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ABSTRACT 

The evaluation of instructional television (ITV) is a 
new field with evaluation methods ■ largely borrowed from other 
disciplines. The evaluatox is not apt to be trained specifically in 
this area but will learn "on the job" while defining his role for 
those with whom he works. The evaluator should consider the following 
recom^mendcitions: (1) Evaluation should be an integral part of the ITV 
project plan; (2) The evaluat^or should help project planners define 
their short and long term goals; (3) Unintended results may be a 
by-product of the project and should not be overlooked; (4) Improving 
the methods of cost analysis is a very important area for future 
evaluations; (5) The evaluator should balance i:hose aspects of the 
program that require definite decisions and those that seem most 
critical ^f or the program's survival; (6) A reasonable time frame 
should be observed and decision deadlines met; and (7) Results should 
be communicated in a clear and useful fashion to the right person. A 
five page bibliography is included together with a variety of sample 
guidelines and questionnaires used in other evaluation studies. 
(Author/^NR) 
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POREWARD 

Tne present essay on evaluation of instructional television 
is meant to be a first step in a process. The whole rapidly growing 
field of evaluation is relatively new, that of ITV evaluation has only 
just begun, l&iesco has fostered tlie use of ITV in a number of educational 
projects throughout the world. Although technology has considerable 
promise for solving educational problems, the cost and conplexity of 
a technology like ITV is such that a failure or even a low degree of 
productivity could mean much to countries investing. Lessons from 
one country can also serve to help others decide vihat role ITV might 
play in their own educational development. As a consequence of such 
needs, Unesco sees evaluation as an iJiqDortant conponent of ITV projects. 

We have been asked to write this essay as a first step in 
the process of creating better evaluations of ITV. Our experience over 
the past ^ve years in a number of evaluations of educational technology 
forms the basis of the present study. Its focus is on the general 
area of what to evaluate. Its audience is those evaluators in ITV 
projects viho have the inportant roie of monitoring the planning and 
progress of technology in their countries. We decided to divide the 
task of writing about evaluation into several steps. For now we 
concentrate on vAiat should be evaluated in ITV projects and not on 
how . The former approach identifies the many aspects of the technology 
that need to be assessed and reported. An attenpt has been made to 
expand the notion of evaluation from one of a testing program for 
classroom teleAdsion to a more conplete inventory of the system^ s 
inpact. In a sense the how or the design and methodology of evaluation 



must come after one decides on what needs to be evaluated. Evaluation 
methods are largely borrowed from other disciplines and applied within 
a framswork ot decision-making about the value of a certain project 
or program* In a sense, these methods are available to the evaluator 
who can identify ;^at he needs to evaluate. His job is to aggregate 
data from various sources in such a way as to render a valid judgment 
of his project. 

We hope that this first step will lead to others - feedback 
from field evaluators on this description of the aspects of ITV 
evaluation; perhaps a field test by several evaluation groi5)S'of the 
aspects discussed. These field tests might be guided by seminars 
within the pilot projects given by evaluation e^q^erts on .the design 
and methods appropriate to the particular aspects chosen. The results 
of the pilot tests ndght then be summed i?) in a more coii?)lete field 
manual on evaluation of ITV that combines both the what and the how . 

We would only add that the advantage of this present approach 
of a modest beginjaing essay, field testing and preparation of a manual 
for evaluators is meant to bridge the gap between the large amount of 
theoretical writing about evaluation and its practice in field projects. 
Eventually we would Jr^pe to add to the theoretical literature on television 
and its evaluation but we can only build a solid theory on the basis 
of practice. Until people can begin to try evaluation strategies out, 
we shall not be able to test our theories against reality. This essay 
hopefully makes the first step of tliis process. 

Emile G. McAnany 
Robert C. Homik 
John K. Mayo 

Stanford, California 
December, 1973 
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I. pVALUATING INSTRUCTIONAL TECHNOLOGy: WE SETTIN G 

L 

Educational Needs and Technology 

Throughout the developing world, educational demand is 
growing rapidly. In Africa and Latin America, where vp to 8S% of 
first grade entrants have left scliool before conpleting six grades, 
and vihere secondary schools and universities have served only an elite, 
<f a concerted effort has been made to expand learning 

opportunities. In Asia, where enrolJjnent pressu;res have been less 
severe, improvements in the quality of education have been en^hasized. 

I For economic, political and moral reasons, most national 
leaders have continued to view schooling as a comer>tone of their 
development policies. Yet they have not haJ sufficient financial or 
human resources to singly expand their school systems in traditional 
ways. As a result, many countries have inaugurated instructional radio 
and television projects. By so doing they have tried to hold down new 
investments in classroom facilities and teachers, relying instead on a 
combination of less well trained classroom monitors and television or 
radio programs. In some countries the primary goal has been to provide 
many more young people with a satisfactory education with no appreciable 
increase <ifi per student costs. In other countries where the goal has 
been to inprove the quality of an existing system, per student costs 
have generally risen. However, it is argued that the use of instructional 
technology may be the least expensive way to improve quality on a 
system-wide basis. 

The introduction of a powerful instructional technology 
such as radio or television customarily demands fundamental changes in 

er|c ^ 



the traditional elements of an educational system. In the classroom^ 
teachers are no longer the sole sources of laiowledge* Finding a. new 
role, one that is conplementaiy to new technology, is no easy task. 
Similarly, the responsibilities of school officials at the regional 
and national levels may also change dramatically when technology systems 
are introduced. The amount of teaching and actual control emanating 
from some central point increases with the introduction of a radio or 
television system and the successfiil utilization of broadcast lessons 
may depend, in turn, on the administrators* knowledge of what is 
h^pening at the local level. In the past a central ministry sent 
out ironthly pay checks and an occasional sijpervisoi ; with a new technology 
the flow of inforaiation from the ministry in the form of program 
schedules, workbooks and announcements to teachers via TV is enormously 
increased. To facil^-tate the transition to radio or television systems, 
new supervisory mechanisms are required. Other needs could be cited, 
but the point should be clear: educational systems using technology 
tend to be more hi^ly centralized and dependent on the integration 

» 

of the diverse parts. This means that technology systems are more 
fragile and need feedbadk for proper coordination. The failure of 
one conponent can undeimine the effectiveness of the entire instructional 
process. 

^ It is the interdependence of elements within instructional 
technology projects and their fragility v^ich make evaluation so 
essential. With good evaluation, the supervisors, program producers 
and planners can identify problems early and make needed adjustments 
when failures occur. By correcting small failures, they can usually 
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avoid larger ones* 

Furthermore, since technology systems are relatively expensive 
investments for a country, infoimation on their level o£ overall 
success is inportant for decision-malcers of other countries as 
well. 

The Types of Evaluation ^ 

^ 1 ^ . ^ Evaluative research is concerned with the success or failure 

of projects. Hov/eW, in this essay we consider only those kinds of 

evaluative research Vluch bear on decisions, either within ongoing 

projects or in the planning of future projects. These include planning 

research, formative evaluation and summative evaluation. 

' / 

Planning research entails the collection of essential data on 
a technology s>''stem t^efore it is iji5)lemented and often before a decision' 
^ is made to undcirtakd) it. Among the most common methods of data 

collection are feasibility studies which survey the key aspects of 
an educational drivirtsia^ specifying the technical requirements for 
the transmissiWi of radio or TV broadcasts and estimating how much effort 
will be needed to develop curricula, prepare learning materials, and 
train personnel.''" Planners often have much information of this kind 

already at their disposal. However, the scope of information needed 

^ J' 

Jto design and implement technology projects may warrant new studies. 

Planners of technology systems may want information on student ability 

and achievement levels in the past, as well as some notion of what 

teachers and students are expecting of radio or television in the classroom. 

Teaclier attitudes toward instructional technology must also be well 
f 

1 Cf* bibliography III. A for exajiples. 
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understood by planners if new classroom procedures and training 
opportunities are to be designed to equip teachers adequately for work^' 
in the new system. 

The list of topics for planning research and the. criteria for 
choosing among them could be extended considerably. In choosing areas 
for planning research, administrators and evaluator^ are able to focus 
their attention on the conponents of an educational system which are 
most fragile and therefore most likely to undermine project effectiveness, 

R)rmative evaluati on is a process of data collection during the 
development of a project so that revisions can be^made to 

{ 

improve its functionjjig. Such evaluation touches decisions at every level 
of a project. Sometimes' project administrators must decide whether 
to depend more on printed materials, cl^issroom teachers, or television 
programs for a given course. Program producers need to know whether a 

4 

given concept was learned satisfactorily and if students are reacfy for 
the next unit of material. Script writers need to know v^ether 
particular lessons were able to attract and hold the students^ attention. 
When difficulties occur in these or other ar^ of an instructional 
technology system, fonnative evaluation's used to diagnose the problem 
and to provide scm indication of what corrective steps are called for. 

Suwnative evaluation is a pixzess of data collection designed 
to provide decision-makers with a more conprehensive understanding of 
how a project succeeds or fails in reaching its goals. It differs in 
time perspective from formative evaluation and is usually aimed at those 
decision-makers who control funds to continue or terminate the project. 
Nevertheless, the two approaches overlap in many ways. The variables 
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or effects studied in both q)proaches may be quite similar. Formative 
evaluations of learning and attitudes often do contribute to summative 
evaluations of the same phenomena. The- most inportant differences 
between formative and summative evaluation pertain to the kinds of 
decisions which each is meant to influence. The need for immediate 
infoimation to guide short range decisions (was last week's concept 
learned so that another one can be taught this week?) justifies a ' 
formative evaluation strategy with its inevitable conpromises in 
methodology. However, when gathered in a systematic way, the data 
from formative evaluations often provide the basis for the longer range 
analyses and conclusions characteristic of summative evaluations. 

The Historical Place of Evaluation 

Although the need for evaluation of various kinds has been 
clear for some time, in practice most educational technology projects 
have done without it. What few evaluations there have been cluster 
into three basic groups. The largest groip has been largely judgmental 
or subjective. An outside expert is asked to examine a project in a 
few days or weeks. He A/isits a few schools, catalogues the hardware, 
speaks to some of the people involved, and sometimes obtains cost 
estimates. Such an approach may prove useful in many ways, d^ending 
on the ability and experience of the expert, but it rarely helps a ^ 
project director to improve his system. A second and more sophisticated 
approach incorporates some attenpt to measure effects. Its methods 
may include conparative before -and-after studies of short term learning 
or, more often, surveys of teachers* attitudes about a course or a 
television series. These results can be fed back to program aSninistrators 
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or producers, but often they, are o£ little use in suggesting pragmatic 
changes and may irritate rather than guide producers. The third and 
rai^st type o£ evaluation consists o£ rigorous attenpts to understand 
the functioning of an educational technology system. Here measures of 
effects are combined with analysis of the processes by which those 
effects were produced. / ^ 

One of the reasons why th^re have been few evaluations of ITV 
systems in the past is perhaps that, in retrospect, there have been few 
users of the, evaluations that have been done, ,and the information 
gathered in even the best formative or summative studies has rarely 
affected decisions in an inportant way. If the results of previous 
evaluations have not been used, why write an essay to advocate more? 
The response to this question ipst await analysis of just why evaluations 
have not been used. 

One can point to three major reasons why evaluation results 
have not been used effectively in the past. First, the focus of 
evaluations has more often than not been defined by the evaluator and 
not the decision-maker; Evaluations are often exercises planned by an 
academic v/ith little ^npathy for the needs of a manager. Secondly, 
because adminisjtrators and program producers are customarily under 
heavy day to da^- pressure to meet schedules, their receptivity to 
learning results from programs broadcast weeks or months previously is 
likely to be quite low. Any information which does not respond to the 
priority concern of getting a program prepared on time is apt to be 
ignored. The third reason why evaluation, results have not been used is 
sinply that evaluations are threatening. While the evaluator may claim 
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that he is objectively evaluating a system and not individuals, those 
individuals justifiably assume tiiat they ^vill be accountable for any 
negative result. Finally, if one adds to the above list of reasoijs 
the fact that evaluation costs money and that eval.uation units must 
usually coirpete for their share of a system's budget, the question 
posed earlier seems well justified: Why do evaluation? 

The fact that evaluation has often been done poorly in the 
past does not lessen its essential inpoftance for educational systems 
using technology. The real challenge is to transform the need for 
evaluation into a commitment on the part of projects and proje^ct 
personnel to use evaluation and to help evaluators inprove their work. 

Evaluators must respond to the concerns of decision-makers 
and report results in ways that bear directly on the latter 's decision 
alternatives. Program producers must be granted more time and encouraged 
to use evaluation results to inprove the quality of their broadcasts. 
At* the same time, evaluators should recognize the real limits of time 
and action of producers, and design their formatfv^e evaluations to fit 
within those constraints. Finally, project administrators must 
introduce a clijnate in which teaching effectiveness has first priority, 
both as a general system goal and as ^ criterion for judging the 
success of the producing organization and its personnel. Too often 
the responsibility, of a production center ends when the program is on 
tape and ready for broadcast. In broadening the concept of producer 
accountability to enconpass what results in the classroom, the administrator 
must reco^iize that success and failure, while not divorced from the 
actions^of individuals, have multiple caascs. Evaluation <^ai) best serve 

,/ 
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^' the principle of accountability only if it is used to infjrove 
program effectiveness and not to lay blame at the feet of the easiest ^ 
target. 

Organization of the Essay 

The rest of this essay examines various aspects of the 
question, what should be evaluated in an instructional technology 
project? Section II describes the first step of the evaluator, 
exploring tlie context, goals, and assuirptions of a project. Section 
III revifev/s a schema for evaluating the achievenient of a project's 
objectives and details a range of intended and unintended outcomes 
that have concerned or should concern evaluators. Section IV 
takes up the difficult evaluative issue of explaining a project's, 
success or failure. Recognizing that a very large array of research 
topics was described in previous sections, Section V describes 
some criteria to help evaluators select among them. Finally, a bibliography 
on eA^luation theory axid methodology is presented in the appendix. 

II- SPECimNG PROJECT OBJECTIVES AND ASSUMPTIONS 

Before an evaluator, particularly an evaluator from outside 
e project, begins his investigation, he must construct a global picture 
of the project. The quality of the data an evaluator collects as 
well as his ability to make- insightful interpretation of it idll depend 
on a thorough understanding of the histprical context, objectives mid 
value assuniptions of the project. 

f 

II). 
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As a first step, an evaluator should familiarize himself 
with the relevant scientific literature. For almost any use of 
instructional technology there are precedents, similar projects for 
which there are available written descriptions , liistories and perhaps 
even evaluations. IVhile no two projects ever overlap conpletely, a 
review of existing information can illuminate possible trouble points 
'l and suggest important research questions. In addition, such a review 

■t can help the evaluator select the most appropriate methodologies for 

2 

his study. 

After preparing himself in this way, the evaluator can 
examine his own project with greater perspective. The first step 
includes three basic activities: specifying the project's internal 
objectives, defi](^g its external objectives, and exploring its value " 
■assunptions and underlying development model. 

Specifying the Project's Internal Objectives 

An evaluator needs to have a clear idea of \^zX. a project is 
expected to accomplish, so that he knows v^at constitutes success 
or failure. However, objectives expressed in planning documents or 
by project leaders aro often vague and without clear criteria (i.e. 
improve the quality of learning, modernize the traditional school, 
x^grade rural education, etc.). In addition, objectives are often 
overly ambitious, a result of the planners' initial i^eed to gain acceptance 
and sipport for their project. ^ 

Oftentimes planners have not adequately thouglit through 

2 Some help for this task can be found in the bibliography jn the 
appendix. 
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their project's objectives. They may even have endorsed a particular 
ii4tructional technology vrithout first focusing clearly on the problems 
it is expected to solve. In some tases, project planners may not be 
willing to specify their objectives for fear they will be held too 
strictly accountable by their superiors. In all such instances, however, , 
the evaluator's need for specific objectives remains and, one way or 
anbther, he must help project leaders to clarify them as best they can. 

The task of transfoming vague or unrealistic project 
objectives into specific ones is not always, easy. Inportant 
steps in the specification process customarily include: 

1. the separation of aiAiguous or confounded dimensions of a particular 
objective into discrete variables (e.g. increasing educational 
opportunity in many ITV projects often does not distinguish between 
opportunity of entrance or equality of learning; the first is' 
measured by enrollment statistics, the second requires monitoring 
learning gains of different social groiq)s) ; 

2. the full definition of each variable in which some change is 
predicted (e.g. defining equality of learning not only in terms 
of academic achievement but also on some standard of success in 
later life) . . 

3. the setting of some clear change criteria for each variable that 
is to be evaluated (e.g. it is difficult to decide whether 
statistically significant learning gains are also socially significant; 
nevertheless, evaluators must set criteria for learning to judge 
success or failure). 

The brevity with which each of tue.^e steeps is stated should not mislead- 
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the reader. IVhile transforming some objectives into specific research 
variables can be relatively straightforward, there are many objectives 
for \diich such transformations are very difficult, if not in5)ossible* 
There may also be some objectives that cannot be measured within any 
reasonable time frame or research bu^-^et. 

There has been an unfortunate tendency among evaluators to 
concentrate only on variables that can easily be measured rather than 
on those that most adequately reflect the true range of a project's 
objectives. Just because variables are not easily quantifiable does 
not mean they should be ignored. Hard to measure objectives 
can' be dealt with in other ways, however. One can often examine effects 
related to a particular objective, even though direct examination is 
not possible. For exanple, if a project's main objective is to curtail 
the exodus of rural school graduates from the countryside, an evaluator 
miglit measure student attitudes tov;ard city life or their occipational 
aspirations as indicators of future plans. Iftider the assunption tliat 
students wha da^intt|nd to move to the city will also be more interested 
^ in visiting it, one Could also look for changes in the number of 

student trips to the city over the period of a year or two. In each 
case, approximation to the underlying variable, future migration rate, 
is achieved. 

Another ^proach to hard to measure objectives requires that 
the planners and evaluators of a project specify the steps required to 
reach an objective. What are the intervening objectives that must be 
reached by a project before it attains its ultimate goals? If one 
wished to curtail the urban migration rate by providing good schooling 

/ 
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in rural areas through ITV, a preliminary step might be to provide 

young people with information via television about eiiployment opportunities 

in rural areas. One could then measure whether or not that information 

\ 

had been effectively communicated. Other such preliminary objectives 
could also be defined and measured. To the extent that they were not 
achieved, the evaluator could infer logically that the ultimate objective 
was also not being achieved. * . ' 

Defining External Objectives ^ ^ 

Nfost ITV projects have some notion, albeit a vague one, 
of their inmediate objectives. More graduates of secondary school, 
inproved mathematics learning, more literate rural dwellers, or nore 
farmers accepting innovative agricultural practices are anong tlie more 
common objectives of instructional technology projects. Yet it is 
clear that these objectives are not ends in themselves; planners assume 
that they will be stepping^ stones toward some longer range development 
goal.. 

In El Salvador, instructional television was introduced into 
all public seventh, eighth and ninth grade classrooms as one con5)onent 
of a general educational reform. Among the expectations of the project 
planners was that many more students would graduate from ninth grade 
and thereafter constitute a trainable labor pool for industry. However, 
the internal objective of producing mre secondary graduates was 
subsidiary to the external objective of accelerating the country *s 
economic development. 

In looking at external objectives, ,the evaluator asks, in 
effect, what will be the societal effect of an educa^tional project if 
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it is successful? IVhat is the developirent model that the planners are 
enploying in their choice o£ project objectives? Such questions are 
rarely the day to day fare of evaluS^ion, yet they are an essential 
exercise for planners. Unfortunately, direct evaluation of such 
objectives is often not possible. They are customarily long term 
objectives, achievable only years or decades hence. Few evaluators 
consider such objectives to be within the scale of their research. In 
addition, societal benefits are difficult to assess from a single 
project. The social outcomes go beyond the particular population 
exposed to a media project and the achievement of benefit's is expected 
to be a consequence of many causes and not just an isolated project. 

Sometimes historical evidence can be used to estimate whether 
or not an internally successful project is leading to given social 
benefits. Assuming that similar objectives have not been achieved in 
other countries, doubts about their fulfillinent in a new project may 
be raised. Thus some have questioijed the evidence for El Salvlidor's 
assunption that more ninth grade graduates will accelerate industrial, 
growdi and economic development. They claim that too little evidence 
si5)porting that assunqjtion exists to justify it as a basis for El 
Salvador's educational teform. 

Neither empirical nor historical methods have yielded very 
satisfactory evaluations of external objectives in the past, both 
because of problems of long time spans involved and the complicated 
relationship between education and society. There is value, however, 
in defining a project's external objectives: it permits project 
planners and those ivlio might fund the project to make some estimates 
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about the likelihood that these objectives will be achieved. This is 
often critical at the planning or funding stage of a project. Thus, 
even though Sesame Street has not recently stressed as a goal helping 
close the learning g^ for minorities in the U.S. , this -was a social 
benefit mentioned when the idea was first being discussed for funding. 
If Unesco funders did not believe that a proposed ITV project would 
acconplish certain external goals claimed for it, like helping a 
nation's industrial growth, steinniing the rural exodus or closing a 
learning gap between social groi5)S, then they would hesitate to back 
sugh a project even though it might acconplish its internal objectives 
satisfactorily. 

Exploring Underlying Assumptions 

Assuming that both the internal and the external objectives 
of a project can be defined, planners may ask an evaluator to go one 
step further in the specification process and ask -him to maker explicit 
the value assun^ptions on which the project rests. If more ninth grade 
graduates are produced in order to attract industrial capital ,| planners 
considering adoption of an El Salvador type project in another country 
may ask if, in fact, that country should place as high a priority on ' 
industrial development. If rural development enjoys a high priority, 
a project which assumes rural -to -urban migration may be a poor one 
to adopt, no matter how efficiently internal and external objectives 
could be met. Again, if an ITV project is capital and not labor 
intensive, *it may on these grounds be less useful for countries with 
large pools of trained teachers. 



15. 

Given that external objectives are specified and the development 
model made e^^plicit, deciding whether they are applicable to another 
country may be a fairly straightforward activity. A more difficult problem 
may be reconciling different personal and social values implicit in 
the design of a media based system. Most countries place a high 
priority on economic development; yet' that is rarely a country's only 
priority or objective • Other objectives, such as increasing national ^ 
pride and loyalty, community cooperation, and the sense of personal 
efficacy, rank high for some national leaders. Yet, accepting multiple 
objectives simultaneously can lead to difficulties. 

Let us say that planners believe that widespread literacy 
is essential for economic development, and further. let us assume 
that televised programs in conjunction with formal classls^are the 
most efficient means for achieving literacy* However, those planners 
may also believe that television as a centrally controlled delivery 
system is counterproductive to an inportant secondary objective ^uch 
as building a higher ^sense of personal efficacy among people. They 
may see that the achievement of one objective may work against the 
achievement of another. In order to maximize both objectives, tliey may 
ultimately elect an alternative method of instruction, one not as « 
efficient in achieving literacy, but more likely to increase personal 
efficacy. 

Decisions about unwanted effects are further complicated by 
the lack of evidence on the social consequences of instructional 
tedbnology projects. In the next section, we consider some intended 
and unintended effects which may be open to evaluation. However, 

'22 



16. 



whether backed by subsequent enqpirical work or not, an evaluator should 
elaborate as best he can the values inplicit in any project he is 
about to investigate. 



III. EVALUATING THE IMPACT OF PROJECTS 

Once having elucidated the internal and external objectives 
of an ITV project, no more iii5)ortant task confronts the evaluator than 
specifying what sorts of questions about that project he will attempt 
to answer. The derivation of such questions is never wholly free from 
outside pressures and constraints. Recognizing that certain kinds of 
research may be forbidden or mad 3 unduly difficult for political or 
cultural reasons and that limitations of time and money may restrict 
the scope of an investigation to but a few of the questions that may be 
in The evaluator 's or his sponsor* s mind, the former must be economical 
with his resources and prepared to concentrate them in areas which will 
allow ;him to achieve the basic purpose of his evaluation. 

Criteria of Achievement of a Project's Objectives 

Evaluation often implies the posing of different kinds of 
questions about a project. These questions ask about the success of 
a project in the £iv^ following ways: 

1. v)hat has been the effort expended and received? 

2. iVhat has been the effectiveness of the project? 

3. IVhat has been its relative effectiveness? 

3 For similar treatment on a more general level, cf. Suchman CCh.4), 
Bibliography, I.i. ^ 
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4. VJhat has been itsx ^fficiency? 

5. IVhat. was the ^process o£ the project's achievement? 

The first fopr^uestions are treated in this section, the fifth in 
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the following section. 

/ 

At the most rudimentary level, the evaluator may wish to 

/ 

verify eirpirically that all project activities are actually taking ' ' 
place. Particularly in media systems that require new efforts to be 
expended at both the national (transmission) and local (reception) level, 
tlie coordination o£ such efforts constitutes a necessary condition to 
the project's success. For exanple, in Mexico's Telesecundaria system 
where secondary schooling is provided rural children via television, 
before assessing the quality of learning the evaluators wished to 
confirm that the television lessons were produced and transmitted on 
time ana that classes were adhering to the broadcast schedule. The 
common experience of evaluators arriving unOTnounced at a school in 
the middle of a scheduled broadcast and finding the pupils still in 
recess or occupied with some completely extraneous activity underscores 
the necessity of this level of evaluation. In media systems purporting 
to serve non-school populations, the audiences are inherently less 
visible and less cohesive than the typical student groip. The monitoring 
of local participation in such instances is necessary to detennine 
whether or not the program is, in fact, reaching those for whom it is 
intended. 
Effectiveness : 

Once the evaluator has established that a particular project 
is indeed reaching its intended audience, other questions may be raised 
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about its effectiveness. Depending on the objectives of the evaluation 
and its scale, studies may be initiated to test whether desired outcomes 
have been obtained. In most evaluations, success has been defined in 
cognitive terms; students mastering new concepts, rural adults learning 
to read, mothers learning about new birth control practices • Frequently, 
other kinds of measures have been introduced to see how audiences 
reacted to the programs they received and whether or not their attitudes 
and aspirations changed in conformity with the project goals • In' 
rare instances Pleasures have been developed to determine if audience 
behavior has been affected in any lasting way by a project. Behavioral 
changes are perhaps the most difficult to evaluate because of the time 
lijTiits inposed on any evaluation. In the case of mothers receiving 
birth control infomation via television, for exanple, one can test 
to see if essential information has been conmunicated effectively and 
if the mothers have been disposed to use it, but many years may be 
required to determine ^^ether such a canpaign has had any lasting 
impact on the birth rate in the listenijig area. 
Re lative effectiveness : 

The task of evaluating a project's performance, whether in 
cognitive, affective, or behavioral terms may be further conplicated 
by tlie fact that no specific criteria of success have been articulated 
beforehand. Planners, administrators, and others involved in an ITV project 
may have quite different ideas about what constitutes success, and 
their judgments are inevitably colored by tlieir unique positions aid 
responsibilities witJiin a project. For this reason, and because 
objectives are rarely stated precisely at the outset, the burden of 
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defining success often regts with the evaluator. 

Although success can be defined in many ways, most media 
system evaluators have relied on one or more of the following three 
criteria: inpact on a target population, injpact on a target population 
vs. a non- target population, and performance to some predetermined 
standard. 

Returning td the exanple of a project teaching secondar>' 
subjects by television, success on the first criterion might ^consist 
of students sijiply being able to inprove their scores on a test 
administered at the beginning of the year and then again at the end 
of the year. Such a before-after design can record the level of 
student achievement with television, but it cannot determine the effect 
of televised instruction per se . * 

To isolate the effect of televised instruction, evaluators 
often try to measure success in conparative terns. Did class A which 
studied witli television outperform class B wh^^-^did not? The validity 

i 

of such a conparison depends on the evaluator *s ability to study 
classes that closely resemble one another on all variables save the 
use of television, i.e. preparation and e^qperience of classroom teachers 
preparation and background of students, community support of the school, 
etc. The random assignment of students to television and non- television 
classes is perhaps the most effective way to insure conparability 
betiv^een instructional conditions, but random assignment is a difficult 
strategy to implement for administrative reasons. 

The tliird and strictest "definition of success is predicated 
on the existence of vje\] defined objectives and performance criteria. 



20. 



Success is here defined as a target population's ability to obtain 

a certain level of knov>rledge, reach a prescribed level of consensus 

/ 

on some subject^ or exhibit certain behavior with sufficient frequency 
to fulfill project goals. While such an approach has proven useful in 
the construction of feedback tests as well as other shorter term 
evaluative instifujnents, project administrators have been reluctant to 
be held accountable for longer term social or even educational objectives 
which may be unrealistic to begin with and thus subject to considerable 
readjustment in' the course of a project. 
Efficiency : 

Even with well-defined objectives, the evaluation of a project 
may still not provide policy -makers with the kinds of information 
they need to determine ii^^results were obtained in the most efficient 
way. In fact, most evaluations of media projects have been of limited 
practical value ^to.^ decision -makers because they have examined the 
effectiveness of only the single approach of media and have ignored 
possible alternative strategies for acconplishing the same objectives. 
When only dhe approach is analyzed, it is hard to be sure that it is 
the most effective one to pursue. Furthermore, even when alternative 
approaches are coiq)ared on some criteria, and one is proved to be si:5)erior, 
it' is still inpossible to decide which approach to adopt permanently 
without considering costs. If a ne\>r approach proves superior to 
traditional alternatives but costs substantially more, it may be 
necessary for the evaluator to consider how the traditional approaches 
rdght perform were the same amount of money to be applied to improving 
them. IVliile it may not be possible to examine thoroughly every 
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possible use of project funds, evaluators should be encouraged to 
investigate costs and effectiveness in terms of alternative strategies • 
Conparative cost -effectiveness analysis has proven to be the most ^ 
useful frameswprk for' carrying out system evaluations of this kind, 

Qioice of Some Areas for Evaluation ^ 

We have stated that the evaluation of an ITV project must 
begin vdth a consideration of objectives. Because most ITV projects 
have multiple objectives - to iiq)rove learning, to extend educational 
opportunity, to interest students in new careers ^ to lower the unit^ 
costs of instruction, etc. - the evaluator must be selective about the 
effects he wishes to focus on. He must also be guided by a well- 
defined set of research questions, recognizing that such questions 
may not be appropriate to the evaluation of other projects. With 
this caveat in mind, it is possible to review the kinds 'of questions 
and variables evaluators of ITV systems have concentrated on in the 
past. 

Student cibitity and achievement : 

Because student learning is a major concern of educators, 
it has almost always been a concern in j[TV evaluations. Two aspects 
of learning^, general ability and achieVenvent, have frequently been 
tested. General ability lests measure such things as students' 
reasoning, languagtj, and problem-solving skills. In the design or 
evaluation of an ITV project, educators may want to know how such 
abilities are distributed within a student population. In many developing 
countries, mrsl children and children from poor families read less » 
well and leaim at a slower rate thiin do urban children and children ^from 
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privileged social groi5)S. This fact can cause problems for the planners 

of new IIV systems and undermine the effectiveness of existing ones if 

evaluation shws that technology systems meant to close learning gaps 

may actually be widening them. Also, when ability tests are administered' 

over a number of years as they were in the evaluation of El Salvador's 
4 

ITV system, they can reveal whether televised instruction has inproved 
students' general abilities and whether or not any equalization trend 
has appeared between advantaged and disadvantaged groiq)S. 

Achievement testing in academic subjects has also been 
undertaken extensively in the evaluations of ITV projects. However, 
when course objectives have not been well defined or when standardized 
norms do not exist, evaluators have often had to develop their own 
achievement tests. This has usually proved to be a %ime consuming 
task, but onel that is vitally iirportant to the assessment of student 
learning with television. With adequate sairpling and control procedures, 
achievement tests have been used extensively to conpare the learning 
performance of ITV and non-ITV students and sych tests should make it 
possible to separate out the unique contributiori^of televised instruction 
to the learning process, although this crucial step has not yet been 
taken in any of the major ITV evaluations. 
Student attitudes and aspirations : 

Evaluators o£ IW projects have customarily paid close 
attention to the attitudes and aspirations of students to see what 
expectations the latter bring with them to school and \\o\^ those 
expectations are affected by one or more years in a television class. 

4 Cf- finaJ evaluation report on El Salvador unc^er Honnjl: et al. in 
Bihiioi^aohy, III. B. 15. 
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The measurement of educational and occiq^ational aspirations as well 
as attitudes toward ITV and schooling in general can help form a conposite 
portrait of a changing student population, IVhen examined over enough 
time and enough students, such measures can also be used to illuminate 
patterns in the relationships between sttident attitudes and background 
diaracteristics , ability levels, and learning outcomes. An awareness 
of changing student values and desires can alert planners early to O 
the conmon problem of sharply rising student expectations and the 
spectre of future disappointment and frustration. With forewarning, 
planners should be in a better position to modify their projects or at 
least inaugurate counselling programs in the hope of stimulating student 
interest in more realistic academic and job opportunities. Unfortunately, 
fe\^ educational planners have heeded th^ results \Axich evaluators have 
presented in this area and, as a result, most ITV projects have propelled 
students forv^ard in the school system with too little concern for how 
they will eventually make use of their schooling. 
Teacher attitudes and performance : 

Although many ITV projects have been justified on the grounds 
that classroom teachers are simply not qualified to carry the full 
burden of instruction, few, if any, have replaced the classroom teacher. 
In fact, most projects have assigned classroom teachers the major 
responsibility for introducing and using television at the local 
lex^'el. For this reason, evaluators h^ve paid close attention to how 
classroom teachers perceive ITV and how such perceptions cliange over 
time. Periodic surveys of teacher opinions have identified- the kinds 
Of teachers who feel they /are having the most success with ITV, and in 
some projects evaluators have been able to relate teacher attitudes 
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toward IT\^ and toward the teaching profession to training levels as 
well as a variety o£ social and demographic variables. 

IVhat actually happens in the television classroom is another 
outcome of growing importance to IW evaluators. It has been hypothesized 
that television teachers serve as models for their counterparts in 
the classroom and that modem teadiing tediniques can be spread ( 
through social modeling behavior over television, but there are few 
agreed ipon criteria upon which to base such judgments and many 
^ differences of opinion still exist about \jhat constitutes good teaching. 

As a first step in measuring classroom teaching behavior, the evaluators 
of El Salvador's ITV system devefoped a teacher observation form.^ 
This instrument illuminated not only differences among teachers in 
terms of time spent lecturing and --working with students, but also the 
"progress" of individual teachers toward the adoption of certain methods 
•advocated by the project's plaipers. 
Administrative policies and costs : 

Far tqo often evaluators have focused their investigations 

on learning and attitude outcomes and have ignored the administrative 

> 

aspects of ITV projects. Yet, when projects fail, it is usually 
because their administrators have been incapable of solving crucial 
problems. Little troubles go unnoticed and are soon magnified into 
major crises whidi, in turn, upset project timetables, discourage 
teachers and students, and eventually inhibit learning. To identify 
problem areas, evaluators have begun to turn their attention to tlic 
administrative history of projects. Using multiple data sources 

5 Cf. Bibliography, II. 9. 
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including archival records, in-depth interviews with project leaders, and 
participant observation, efforts have been made to illuminate a project's 
major decisions as well as the crises that hinder its growth and 
effectiveness. Such histories can help guide decision-makers and 
provide useful information to other countries' planning to inaugurate 
similar projects. 

* 

Cost analysis, as we have suggested in the previous section, 
is another area that has been sorely neglected in most IIV evaluations. 
Budgets constrain all ITV projects, but few planners or administrators 
are provided with the kind o£ information needed to make better budgetary 
decisions. Problems of estimating true costs, of describing cost 
functions and their properties, and of introducing methods for coping 
with the temporal structure of IW finance are just beginning to 
interest economists and such topics will undoubtedly become more 
important in future ITV evaluations.* 

Long Range Effects and Unintended Consequences of IW 

There are unquestionably many other intended and unintended 
outcomes of an ITV project that help determine its ultimate success and 
significance. Even though an evaluator must be concerned primarily 
with the stated or inferred objectives of a project, he may still wish 
to devote some of his time and money to such neglected problem areas 
as the demands ITV places on teachers and school administrators, the 
relationship of project goals to outside economic realities ,and 
constraints, and ITV's inpact on students and their families. Unfortunately, 
interest in such areas is relatively new, critical variables have not 

* Cf. Bibliography, II. 6. 
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been adequately definecf, and few enpirical studies exist to guide new 
research. Despite these obstacles, however, evaluators are increasingly 
turning their attention to the problems of identifying and measuring 
some of the longer range effects and unintended consequences of ITV. 
Effects of ITV in the classroom : 

Beyond the attitudes and interactions of. teachers and 
students which have interested evaluators of ITV projects in the past, 
it has been hypothesised that the introduction of classroom television 
may have longer term effects on the quality of social relations within ^ 
schools. In the traditional school, the classroom teacher was the 
dominant force. He or she set the pace of instruction and was responsible 
^ for vAat was taught. With ITV, however, the typical classroom teacher 
no longer exerts such exclusive control. For the first time center 

J* 

stage must be shared with one or more "master" teachers whose lessons 
arrive at prescribed intervals via the television screen. The classroom 
teachers are forced to schedule their activities around such broadcasts* 
How does such an innovation and the changes in teaching styles it 
demands affect the authority and self-esteem of the classrooin teacher? 
Are students more or les$ attentive to their classroom teachers when 
the latter act as monitors instead of subject experts? If, as has 
often been suggested, ITV permits school administrators to restructure 
the school day and/or to increase greatly the student -teacher ratio, 
will teachers be able to adjust to such changes, maintain respect, and 
still interact with students on a personal level? The. answers to such 
questions have inplications not only for the achievement of short term 
ITV objectives, but also for the survival of the school in its present 
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form - a subject which itself has come under heated debate in recent 
years. 

Effects of ITy on a school eye tern : 

The long range effects of an ITV project on other aspects of 
a school system may also be of interest to an evaluator. Recognizing 
that a powerful innovation such as teleAdsion inevitably requires some 
bureaucratic reshuffling, teachers and administrators may not be 
willing to implement policies that seem to undermine their freedom of 
action. In most traditional systems, classroom teachers and their 
principals enjoyed considerably autonomy. ^Such autonomy was usually as 
much a result of their school *s distance from the capital city and its 
neglect by higher officials as any other factor. However, with the 
introduction of ITV, teachers and local administrators are customarily 
subjected to strong centralizing policies. Decisions affecting all 
television classes are formiilated in central government offices and 
mandated throughout a school system. To insure the effective use of 
television at the local level, school supervision may also be reformed 
and expanded. While such change^ are devised to xxpgrade the efficiency 
of an instiTictional system, their inplementation often breeds teacher 
resentment. Teacher resistance to ITV in El Salvador, Mexico, and 
elsewhere has been demonstrated by heightened union activities and 
claims that television has been purchased at the price of inproved 
teachers' salaries and working conditions. The inpact of teachers' strikes 
and other political activities on the development of an ITV project 
are inportant developments whose short and long range consequences an 
evaluator cannot afford to ignore. 
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The inportation of equipment and technical assistance as 
part of any ITV project may also have long range consequences on an 
educational system's personality and development. Virtually all ITV 
projects in the developing world have relied vpoa heavy doses of 
foreign hardware and expertise to get started. While some countries 
have been able to manage such resources conpetently, many others have 
either lost control or sinply forfeited decision-making power to 
outsiders from the outset. The different patterns of foreign resource 
management and their relationship to institution-building activities 
within developing countries are subjects which should also concern an 
evaluator interested in assessing the effectiveness and survival potential 
of any ITV project. 
Effects of ITV beyond the school : 

Beyond the school, there isj a wide range of individual, ' ^ 
institutional, and social outcomes of any ITV project that could occupy 
an evaluation team for many years. In charting the vast unexplored 
territory beyond the immediate objectives of a particular project, 
the evaluator must be cautious in his ^proach and aware that the 
answers to many fascinating research questions are likely to remain 
outside his grasp. Nevertheless, with good theory and careful planning, 
he may be able to make significant progress in extending the boundaries 
of his evaluation and thereby make inportant contributions to knowledge 
and to the area of evaluation itself. 

Neither ITV projects nor the larger school systems of which 
they are a part exist for their own sake. At the root of a nation's 
educational policies are basic beli(»fs concerning the inportance of 
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schooling to economic productivity, social haimony, and individual 
happiness. Until recently, the faith in schooling as a social good 
was virtually absolute. Hoi^ever, 'the skyrocketing costs o£ providing 
even a few years of basic education to a nation's youth coupled with 
the radical questioning of established educational institutions and 
values have pronpfed both decision-makers and scholars to review their 
assumptions about schooling. For the first time, the promised benefits 
of schooling are being critically evaluated. The motivation behind 
such analyses and their content are directly relevant to the assessment 
of the long range outcomes of any ITV project. 

One of the most fertile topics for research in recent years 
;has been the linkage between schooling, job opportunities, and economic 
development. From a variety of disciplines has emerged the rather 
startling conclusion that the correlation between success in school 
and success on the job is not nearly as high as was once believed. 
Furtheimore, there is a growing consensus among radical economists of 
education that the investment in certain kinds of educational innovations 
such as ITV has only aggravated an already intolerable situation by 
stimulating student desires for more schooling and preparing young 
people for careers that are sinply not open to them. Critics warn 
that the distortipns caused by excessive investments in schooling 
exacerbate existing inequities in a society and make needed structural 
reforms that mucli more difficult to achieve. It is far more difficult 
to convince young people to remain in the countryside, they argue, 
when a country's educational policies are drawing them inexorably to 
the cities. 
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If a long range result of an ITV project is to draw talented 
young people away from their rural origins,- it will inevitably 
affect the social fabric of the rural comnunity and, perhaps, the 
relationship between parents and children. In traditional comnunities 
and cultures, independence from one's family was frowned upon and young 
people v/ere taught to abide by the authority of their parents. However, 
when young plople enter school, they are confronted by new kinds of 
expectations and rewards, both intellectual and social. When school 
behaviors and values clash with traditional family norms , conflicts 
may arise between students and their parents vis-a-vis the fomer's 
ambitions and plans. Although some friction between the student and 
his family is inevitable and may actually be iJi5)ortant to the young 
person's development, the radical critics of schooling question who 
really benefits vdien the student ultimately decides to pursue a life 
in "the modem sector." Is it the student viio has succumbed to the 
attractions of urban life and the possibility of a better paying job 
away from his family, or others who control the wealth of a country 
and who count on the school system to socialize young people for work 
in an economic envirdtlnent which they control? Questions of social equity, 
equal opportunity, and the relationship of schooling to development have 
become increasingly iirqportant to the evaluation of ITV as well as other 
educational innovations. 
Wlio benefits? : 

In designing an evaluation, the investigator begins with an 
independent variable - in this case tele\lsion - and, then defines and 
measures certain effects of that variable. With all the measures of 
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effects described in this section - cognitive, affective, and behavioral 
the question can be asked: \iho benefits? Do students with high ability 
scores learn significantly more than those with low scores, urban pi?)ils 
more than rural, boys more than girls?^ If so, v*iat are the implications 
for the development of television lessons and the materials that often 
acconpany them? Are the gains in one subject notably greater than 
those in others? Are achievement patterns closely related to ability 
scores and demographic and social indicators? 

If an evaluation is extended over a number of years, it is 
frequently possible to determine whether television has had any 
cumulative effect. To the extent that conparable measures are used 
from one year to the next, it is possible to see v^iether courses are 
operating more effectively or. less effectively than before and whether 
a television system has altered in any. way the relative achievement 
levels among ability groups. For example, did the addition of television 
help to -equalize the achievement of rural and urban schools? Lower 
ability and higher ability students? In other words, vAat is the 
evidence that ITV has had any long range inpact on a school system, 
and how can the answers to such a question be explained? 

IV. EXPLAINING ACHIEVENEOT OF OBJECTIVES: PROCESS EVALUATION 

Let us assume that an evaluator succeeds in measuring 
the achievement of the defined goals of an instructional technology 
project as outlined in the previous section. If we take a narrow 
view of^the evaluator's task, we might be satisfied that he has come 
6 Cf. Bibliography, III.B, 15 and 16 for some treatment of tliese points. 
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to a judgment about the value o£ the project (if nis task is sumniative 
evaluation) or about short range success or failure (if his task is 
formative) • His next task is to detemine why the project succeeded 
or failed. For this he needs to understand better the process the 
project followed in its development. 

C onditions of Success and Failure 

Evaluations are more often concerned with examining the 
process of a project when the're are failures than when there are 
successes because of a need to make someone accountable. Thus the 
enphasis here is usually in temns of wliy projects fail than why they 
succeed. The failure of ITV to bring about desired or promised 
effects may have several explanations. The effects promised may have 
been exaggerated, or the objectives may not have been achieved for 
reasons that need little sophisticated measurement. For exanple, an 
ITV project may never have reached the production stage, another 
project may never have gone beyond a few pilot classes, and a third 
may have been terminated after a trial period, all for reasons that 
stem fron^three closely inlierrelated environments: the classroom, 
the educational administration, and the political system. 
The clasoroom environment: 

The conditions one usually examines in a classroom environment 
to explain success or failure of an ITV project are most often related 
to the project's learning objectives, A first step is to describe as 
con^jilctely as possible how the unreadied objective u.is to he achieved. 
Foi oxiimi)lc, Jet lis say that a particuLu project usl> television in 
Cur luiKt^o!' \ iV> classroom teacher an;" rt^Jent uorklu^ol^ leat^ math. 
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In 'a gljven unit o£ material, each conponent (TV, teacher and workbook) 
is responsible for some proportion o£ the instruction. At the end of 
the unit, a test reveals that relatively few students learned what was 
expected. The evaluation outcome is straightforward - the system 
failed. The evaluator's job is now to understand why the failure 
occurred and to propose corrective measures . 

The evaluator may first want to review the conditions that 
were believed to be necessary for successful learning... Certainly 
students had to be in attendance throughout the unit. They should 
also have been sufficiently nourished, not overtired from work after 
school, motivated for learning, and adequately prepared in math to 
understand the concepts being taught. Teachers should have arrived 
on time, maintained discipline in the classroom, and taught their share 
of the instructional load effectively. Television sets should have 
been \irorking satisfactorily and lessons would have needed to be 
broadcast on time. Otiier preconditions to successful learning could 
surely be suggested. However, assujning that this list is exhaustive, 
the evaluator must determine which ones were not met. Many in the 
list can be eliminated with minimum effort. The evidence of technical 
failures - programs not broadcast, poor distribution of broadcast 
schedules or workbooks, and inadequate functioning of television sets - 
can be checked through school visits. 

Conf)aring achievement in well-equipped and poorly -equipped 
schools can provide e\ddence on the physical learning environment. 
Nfea^ures of student attitudes toward scliool and mathematics may reflect 
motiv'iition problems. Con^^Kirisons a^r.ong student^ of different social 
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backgrounds may prove useful in exjiinining the influence of students' 
background characteristics oa learning. 

After analyses of this sort, an evaluator may conclude that 
the television lessons did not present the concepts clearly or that 
: teachers did not answer students' questions adequately. However, 

finding fault is rarely useful unless it can be acconpanied by 
• constructive suggestion. - l"he evaluator may be forced one step baclacard 
to begin explaining failure at another level. What were the conditions 
precedent to the production of high quality telelessbns: sufficient 
personnel, sufficient facilities, sufficient time, sufficient training, 
sufficient knowledge about the audience?- What were the conditions 
precedent to effective teacher participation: positive attitudes toward 
teackSig and television^ sufficient training, sufficient knowledge of 
how to work with TV, sufficiently low student/teacher ratio? IVhat . 
were the conditions precedent to the preparation of adequate workbooks : 
sufficient coordination with teleteachers , opportunity^ for pretesting, 
sufficient trained personnel? 

Thoughtful analysis of these prior causes may turn up 
additional reasons for failure. Two constraints on this sort of 
analysis are clear, however. First, if the evaluator' s task is "formative, 
hp cannot possibly hope to analyze all the^pofe^ntia] sources of failure 
and still provide decision-makers with the infoimation they need to 
rmet production deadlines ♦ Second, the evaluator must try' 
to distinguish between conditions precedent to learning that a project 
manager can and cannot control. Even though an evaluator should 
invostigatc vvhether the differences between niral and iirb.-m settings 
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affect learning- from television, no project manager can directly 
change the poor conditions of the rural areas. His solutions may 
■ involve reversing patterns of resource allocation to favor rural areas 
or simply a recognition that some failures are just not soluble by 
educational decision-makers. 

For the sunmative evaluator, many of the detailed analyses 
of particular course failures will be of less interest than broader 
studies of conditions precedent to success of a whole system. Describing 
all the conditions precedent is a conplex and perhaps an impossible 
task. As a result, the summative evaluator may be unable to sdt 15) 
any exhaustive model of a project; rather he must explain success or 
failure somewhat intuitively, depending on his and other obs.ervers' 
knowledge of a project. ^ 
TYie administrative environment : 

Evaluations are often so focused on the quality of the TV 
lesson or the amount of learning gain that students register during a 
.semester with ITV that they may fail to see hoi^ the overall project is 
coping or failing to cope with keeping itself going: Yet instructional 
technology systems are extremely complicated to initiate and maintain. 
^kly countries may not only lack technical production and broadcast 
personnel but may not have tlie management capacity for running a major 
technology project. Instructional systems that use technology are 
much more dependent on coordination of a series of elements than 
traditional teaching systems. Consequently, the chances of failure in 
one area affecting all areas is greater. What happens, for example, 
if there is a power failure at .the transmission center? Are classroom 
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teachers trained to take over and teadi the lesson? How are schedules 
rearranged? What i£ (as is coiranon in many ITV projects) printed 
niaterials for the lessons - or even the schedule of broadcast programs - 
do not arrive on time? What happens to instruction if the inspectors 
cannot visit outlying schools in their district? How much do breakdowns 
in local receivers interfere with instruction and learning and how 
efficient is the maintenance service in repairing them? 

Historically, project administration has remained largely 
outside the scope of most evaluations because most evaluators have 
been reluctant to consider it, preferring instead to concentrate on 
the quality of broadcast lessons (\dthout a clear objective for the 
lesson being defined) or on the creation and administration of learning 
tests. 

The iii?)ortance of administrative evaluation is illustrated 

n 

from recent studies in Mexico. There two radio projects were evaluated 
over a six month period. During visits to a majority of the radio 
school sites, it was discovered that almost half the schools did not 
have an operating radio at all! To evaluate the quality of programming 
or the effect of radio instruction on students who miy not be receiving 
programs in most cases is a waste of an evaluator's time. In both 
cases, the evaluators pointed out in their studies that the administration 
was not able to insure the system's continued operation. 
The political environment: 

The political conditions in whicli projects operate can also 
determine their success or failure. Television is an expensive 

7 Cr. Bibliography, 1 1 1. B, 19, 20. 
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technology' and often involves spending largo amounts o£ public money 
on hardware and personnel. It also involves access to a large audience 
via a power%L means o£ conmmication. Moreover, it deals with education 
and the socialization o£ young people in the values o£ their society. 
For these and other reasons, ITV projects are politically sensitive and 
subject to many di££erent stresses that n^y a£/ect 'the achievement o£ 
its stated objectives. Since ITV projects o£ten demand the import 
of foreign hardware as well as technical assistance, to what extent is 
, ' a project affected by the political pressure of the donor agency or 

country or a multinational corporation that may be selling hardware? 
How can an evalCiator estimate the extent to which the values and 
models of schooling from out-side the {country affect the plans of the 
indigenous ITV groi?)? Who makes the essential decisions for the project 
and how do these affect the outcomes? Indeed, \im are the goals and 

4 

objectives set for the IT\t<project in the first place? All of tKese 
are questions touching on the political conditions of a project and 
should be taken into account by an evaluator who wishes to be complete 
in his explanation of project performance. 

Evaluation's Interdisciplinary Approach 

To explain the success or failure of a project using 
instructional technology evaluators need to pay attention to a much 
wider range of factors in systems than has been their practice. School 
and class roojn^dlninistrative, and political factors can dramatically 
affect tlie learning outcomes of instructional technology. An ■ 
cvaluStor must understand administrative and political processes of a 
system as well as measure its .^peci lie outcomes. This exjjanded notion 

er|c ' 
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of evaluation also calls for a team approach to evaluation where people 
from a number of disciplines will attenpt to monitor different dimensions 
o£ a project. 

V. DECIDING WHAT TO EVALUATE 

The evaluation of instructional technology projects is in 
many ways a new concern for education and communication researchers. 
If there ivere a long histoiy of research in this area, evaluators 
would be tenpted to look for a cook-book approach v^ere appropriate 
procedures ^re laid out for them. This is not the case for evaluating 
large technologies, however, v*.ere relatively few e^qperiences have 
been recorded for others to follow. There have been a few evaluations 
undertaken - in El Salvador, Mexico, the Ivory Coast, and Samoa - but 
the present authors, having participated in some of them> realize 
that they constitute only a beginning and not a final guide to action. 
How, then, is the evaluator to decide- what he will evaluate in a 
.particular .project? . — • 

J 

t 

The Funding Source 

When funding is made available for an evaluation, we presume 
that some needs have been expressed by the funder. The U.S. government 
was interested in determining whether or not El Salvador's use of 
educational technology would be a good investment for other countries 
and therefore siq)ported a summative evaluation of this project. Because 
cests of such systems are so large, other international agencies have 
similar questions. Thus governments as well as international organizations 
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such as the World Bank and Unesco are interested in stuping the educational 
effectiveness of the instructional technology projects. The French 
goveriunent funded an evaluative study of its Tele-Niger project after 
six years of operation in the field. The Mexican Secretariat of Public 
Education helped fiind a study of its Telesecundaria project five 
years after it got under way to know how it was functioning and whether 
evaluation could 'help improve its operation or suggest new ways to 
reach rural secondary students. There are many other exanples of 
governments and international lending agencies showing keen interest ^ 
in knowing how educational technology is working. 

The problem for the eyaluator is that even vrtien a funding 
agency is willing to invest a considerable amomt of money in assessing 
a project, the agency may not really know what it expects from the 
study. There may be unrealistic expectations as to what an evaluation 
can demonstrate. Evaluators themselves may not realize the relatively 
primitive state of the art in the field or, in their eagerness to 
become involved, overstate what they can acconplish. An evaluator 
mustTre"clea^^ 

define realistically what a funding agency can expect from him. If 
this is not done, disappointment may result and efforts to develop a 
better evaluation model may be undermined. 

Although the evaluator must seek some general guidance from 

i ^ 

his sponsors about what is expected of him,\ he should not expect a 
blueprint for his work. Needs may be defined in terms of ^efficiency - 
"Does ITV really ^olve the problem of significantly increasing secondary 
schooling at lower, "unit cost tJian traditional methods?"; and/or in 
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quality terms - "Iv'hat can you do to help make the ITV programs better?" 
The evaluator needs to assess and define the goals o£ the funding sourceV 
as best he can and see how they fit in with his own ideas of what the 
problem is and what the needs are. He cannot only consider the 
source of funding for guidance in choosing an evaluation focus if 
the users of his evaluation are different from the funders. 

• Potential Users 

The question, 'nvho will use my evaluation study and how?" 
is perhaps tlie best guide for evaluators. Unfortunately, this question 
has not often been ask^d and the resulting underutilization of 

evaluation results is the fault of both evaluators and decision- 
,8 

makers. On thp part of evaluators, the problem has often been that 

* 

they decided unilaterally what aspects needed evaluation and, once 
their work was finished, reported tiieir results in long and jargon- 
filled documents for busy administrators or producers v^io had no time 
to read them. Evaluators often have not assessed real needs where 
^^'cir work might make a useful contribution ^d have ignored the - 
- ^ necessity to communicate this knowledge in a manner that will most 

likely reach the user. ^ On their side, decision-makers can be faulted 
for viewing evaluation as a necessary evil or, worse still, as a 
public relations am of their projects. They may have no intention of 
^ heeding the evaluator 's findings, especially if those results are the 

least bit critical. 



8 Although Stufflebeam (Bibliography,. 1.2) speaks of evaluation keyed 
to decision -making, House TBibliography, 1.2) points to the large gap 
ben^^een the ideal and the real usefulness of evaluations to decision- 
makers in education. 
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How can such problems be resolved? First, the evaluator 
must try to target his work toward real information needs • .This 
means he must work with potential users and see what information they 
must have to make decisions. In 'formative evaluation strategies, work 
might concentrate on a key policy area. If, for example, ITV was 
adopted on the premise that it would provide secondary education for 
rural youth and thereby decrease their desire to migrate to. large urban 
areas in pursuit of post-primary schooling, an evaluator might base his 
shorter-term study on the problem of why rural primary graduates migrate 
and, on a longer term basis whether, indeed, ITV has helped resolve 
this problem* An evaluator wanting to create a learning feedback system 
for producers of the TV lessr >s may first have/"^ show produ cers how 
their prograins relate to learning and haw improvement in learning is 
tied to inprovements in production quality,^ Furthermore, producers 
often have no clear learning objectives for their TV lessons and so 
have no way of incorporating feedback results into their ^ 
production. In El Salvador, it took three years before a seminar in 
formative evaluation and definition of learning objectives, with 
production and evaluation teams participating, brought some common 
understanding to the task of testiiig and production. 

Ofterf overlooked in the well-meaning efforts of international 
agencies to generate evaluations is the problem of doing useful 
evaluation. ' Part oif^^'the effort to promote better evaluations must 
focus on tlie decision -maker and his education as an information user. 
An evaluator ma^t realize that political and administrative factors 



9 Cf. Bibliography, IV. 25. 
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p^ay a predominant part in decisions along witli personal pressures 
on \|:he decision-maker. In addition to doing research, the evaluator 

c — ^ 

must insure tliat his results are given a hearing by the decision -maker. 
This may call for a reorientation of the administrators' approach to 
problem solving. In so doing, the evaluator is not trying to personally 
direct decisions but rather he is seeing to it that evaluation results 
at least enter into decisions. This process demands that the evaluator 
work closely with those whose decisions he is trying to provide 
information for, be they administrators, program producers, supervisors, 
or teaclier trainers. 

lite Constraints 

ITie evaluator must work under a numl)er of constraints. The 
irost coinmon, although not perhaps the most critical, problem is money. 
Many IIT projects have acceded to pressures from funding agencies and 
created evaluation units , 'but have not then provided them with adequate 
budgets. There are numerous examples of IIV projects with a number of 
teachers assigned to evaluation. Lacking both funds and training, the 
teachers' work is often limited to filling out forms on the ''quality^' of 
tJie programs or to observing classrooms (usually classrooms in the capital 
of the country where conditions are quite distinct from most other 
parts). Miat is oftQn not understood by fimdors is that the training 
r-ri r financing of field workers costs money. Communic^itions , transport, 
.juk' : . fHtinj: all add to the cost of evaluation. An ovijJuator mu:^t 
n ' Kv »M ! ucJ;;ct hetorc planning an elahorate study tint jn.iy be iuTjiossihlL- 
t' ^i'*? ' n ..M)iis. Hiirealistic plaji^ ire not uiaoiWn ruiuny \'X\ 



49 



43. 

evaluators who may be restricted to their offices by lack of operating 
funds. Their jobs often become a kind of fantasy of what they would 
like to do instead of what they can achieve in their circumstances. 
If the evaluator has a clear idea of the goals of his work, he should 
calculate exactly what he can acconplish given liis budget, or fight 
to increase that budget to accomplish what is necessary. 

There are constraints in both number and training of evaluation 
personnel. A ministry of education may have a number of teachers on a 
payroll that can be loaned to an evaluation unit, but without at least 
one or t\>ro well-trained evaluators, the added numbers will do little 
good. The lack of well trained people may be tlie key constraint in 
limiting the effectiveness of most ITV evaluations. 

The final, basic constraint is time. An evaluator trying 
to plan his work must set deadlines and stick to them if his findings 
are to enter into the decision -making process. Too frequently evaluation 
is allowed a leisurely pace not permitted in other parts of the system. 
This may give the inpression that evaluation is not to be taken 
seriously and has nothing to do with them. Producing valid and reliable 
evaluation infomiation on schedule is one of the most difficult tasks 
of the evaluator. The evaluator must be a careful organizer both of his time 
and tliat of his staff. Some sort of adaptation of management planning 
mc tiiods should be part of his repertoire of abilities if he is to 
iiK^et deadlines. Too often we hear of an ambitious school testing 
program undertaken with enthusiasm but without sufficient planning. 
Delays, oversights of coding or analysis costs, overestimations of 
abiljt>- to get things done, all lead to long delays that make results 
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of historical value only and thus confirm the suspicions of decision- 
makers that evaluation is a useless academic exercise. A small 
evaluation effort carried out well with relevant information delivered 
on time should convince a decision-maker that evaluation can be an 
integral part of his project and one that should be paid attention to. 

A Practical Decision 

Even when all of the above factors are taken into consideration, 
it is still the evaluator who must make tlie practical decision of what 
to evaluate. IVhat guideliijes might he look for in this decision? Other 
evaluations will obviously guide hini to some degree since he might feel 
more secure doing something that others have successfixLly done before 
him. This might lead him to measure learning, or attitudes, or the 
effectiveness of a pretesting system, or the creation of a feedback 
mechanism for monitoring student progress on a unit of a given curriculum. 
The literature to guide the evaluator of instructional technology 
projects is relatively scarce as the bibliography attests, although 
there is a. large literature on curriculum and school evaluations that 
can give some guidance. 

The essential point is that evaluations need to cover new areas 
that are difficult and sometimes sensitive. More needs to be done with 
cost measurement and cost models, for example. The examination of the 
affective area of learning with ITV should also be a foais of new 
evaluations even though it is clearly understood that measurement is apt 

I'l Cf. Bibliography, II. 

11 Cf. Bibliography III. B. 16 and 11.6. 
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to be veiy primitive at first/ The models and cultural heroes 
presented to student audiences need to be examined critically vis-a-vis 
their inpact on the learning and cultural objectives of a project • 
If an ITV project proposes to encourage students to remain in the rural 
areas but reinforces urban models in its TV lessons, the evaluator 
should be able to show the contradiction between m^ans and ends. 

The. administrative history of a project is a new and 
difficult area for evaluation, but a vital one for most large technological 
^ projects. The iirpact of an ITV project on the social and economic 

standing of its students is a common argument in selling the project 
to the public, but few evaluators have had the^time or inclination to 
critically examine the achievement of these goals. The longer run benefits of 
ITV are some of the most crucial ones, but often evaluation of them is 
discontinued long before they can be measured or reported. Since 
costs are heavily concentrated at the start of an IT\^ project, these 
loom large in conparison with rather insignificant short-tenn effects. 
• Finally, because evaluation is often defined by those who expect some 

specific problem to be solved, evaluators are tied to testing foT 
tliese effects and are not billowed the freedom to look for unexj^ected 
effects. All of these areas are relatively unexplored, especially 
witliin IT\' projects, and thus detailed guidance is not available. 
It may be tliat those with more experience should be the ones to undertake 
these more challenging areas uliile otliers perfect tlieir skills and 
gain experience in applying the better tested methods, 

12 U. Ki*'Moj:iaphy, IV. 21. 
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VI. CONCLUSIONS AND RECONWENDATIONS 

\ 

We have presented in these pages a great many things for 
tlie evaluator to do, but we have not tried to push him in one direction. 
■ i-valuation is a relatively new discipline, and it has not developed 
for itself an ijnage; much less acceptability in many surroundings. 
Somewhere between an art and an applied science, evaluation and the 
• eValuator mean many tilings to different people. The evaluator is most 

likely to be someone trained in anything but evaluation, a psychologist 
or sociologist, perhaps, or an economist, engineer, or teacher. He 
will have to learn his nev\' metier on the job most likely and define 
his role for those he works with. 

The role of infomation in guiding decisions is not, of course, 
new to human behavior. But in large and complex institutions where 
many people must cooperate and many factors be made to mesh to achieve 
a common goal, there is critical need to have a system of feedback to 
help correct and adjust the institution, i-valuation is not only a 

c 

mechanism for gathering the appropriate information in the most 
^ reliable way but of seeing tq it that the information is " communicated 

to tiiose who will make decisions.' The evaluator, or the person who 
coordinates the evaluation, must act both in the rjjle of objective 
social scientist in gathering information, and skijlled manager in 
seeing that information reaches persons who most rf'eed and will use it. 

We might end tliis essay with a summary of what we have said 
to the ^valuator by way of some brief recommendations: ^ 
(1) f valuation ought to be an integral part of. the ITV project plan 
itsell; tli'dt way, it can grow v/ith the project and not be added 
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on as an afterthought; 

(2) The evaluator sliould help planners to define their short and 
long term goals, specify internal and external objectives and 
clarify value positions T>rithin the project plan; 

(3) The evaluator should not overlook those aspects of the project 
that go beyond defined goals and which are related to unintended 
results; 

! ^ 

(4) Inproving the metliods of cost analysis is one of the most 

inportant areas for future evaluations; 

(5) In choosing what aspects of the project to evaluate, the evaluator 
should balance those that seem most urgent for decisions witli 
those that seem most Critical for the survival of the project; 
the two are not always the same;, 

(6) Although the constraints of money, time and trained personnel 
are all critical, the evaluator has most control over time and 
should try to work within reasonable time frames and meet deadlines 
inportant for decision -making; 

(7) Communication of results to those who can use them in a way in 
which their use is most assured should be as inportant to the 
evaluator as the careful gathering and analysis of his data. 
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APPB>IDIX : BIBLIOGRAPHY 

Note: The purpose pf this bibliography is to be illustrative of sources, 
available in ^lish, that form a background to the present essay. The 
essay attenpts to synthesize the relevant literature and put it. into a 
practical framework of action for the field evaluator. , If he is already a 
trained researcher in a social science discipline, he can put his knowledge 
to work with some help from additional reading; if he is not trained in 
research, this bibliography probably will not be of much help. It is not 
meant to be exhaustive in any area except in the precise one of the* essay's 
subjoct, evaluation of ITV in developing countries.' He i-e we have searched 
the published sources rather thoroughly and have found little available. 
Hence our hope that this essay will help to fill a void. Other, more 
general areas like evaluation methodology or educational research have 
extensive bibliographies, of which only a small sampling is included. 
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General Evaluation Methodology 

1. Suchman, E., Evaluative Research: Principles and Practice in Publ ic Servic 
and Social Action Programs. New York: ^^Jssell Sage tbundation, 1967. 

A basic well witten text oh evaluation, using exanffles from health 
projects in the U.S., but presenting principles with general applicabilil 
A briefer and more recent paperback book of the same genera] kind is 
Weiss, C.H. , Evaluation Research: Methods for A ssessing Program 
Effectiveness , hnglewood ciitts: Prentice Hall, i<)7^ — — 

2. Some otlier exanples from the growing field of evaluation literature: 

Bloom, B.S., Hastings, J.T. , and Madaus, G.F., Handb ook on Fonnative and 
Summ£itive Evaluation of Student Learning . New YorTcl — NlcGraw Hill, 1971. 

Caro, F.G.' (ed.}. Readings in Evaluative Rfesearch. New York: Russell 
bago Foundation. l^TTi 

House, E.R. (ed.). School Evaluation: T he Politics and Process. Berkeley 
Calitomia: McCutchen, 1973. — 

Ihis book stresses the problems witli the many large-scale o\aluatiOns 
of schools in the U.S. Gives -perspectives on the areas of evaluation 
.outUde thai of student learning, , 
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Rossi, P., and Williams, W. , Evaluating Social Programs: Theory, 
Practice and Politics > New Yorlc: Seminar Press, 1972. 

Stake, R. , "The Countenance of Educational Evaluation", Teachers 
College Record 68 (1967) , pp. 523-540. 

Scriven, M. , "The Methodology of Evaluation" in R.E. Stake (ed.). 
Perspectives of Curriculum Evaluation . Chicago: Rand McNally, 
1967. AERA Series on Curriculum Evaluation, no.l. 

Scrivon, M. , "Goal Free Evaluation", in House (ed.). School Evaluation, 
op , cit . , ch.24. 

Stuff lebeam, D.L., et al.. Educational Evaluatiofi and Decision 
Making . Itasca, Illinois: Peacock Press, 1971. 

Wittrock; M.C., and- Wiley, I).E. (eds.), The Evaluation of Instruction: 
Issues and Problems . New York: Holt, Rinehart and Winston, 1970. 

II. Specific Evaluation Metho4ology 

3. Bloom, B.sT, A Taxonomy of Edi^ational Ob jectives : Handbook I : 
Cognitive Domain . New York: David McKay, Inc^, 1956. - 

An early attempt to identify different kinds of learning in the 
classroom. 

4. Cajipbell, D. , and Stanley, E., Ex periineiital and Quasi-e^erimental 
Designs for Research . Chicago : Rand- McNally, 1966. ' 

r 

A classic discussion of design problems in social research. 

5. Cronbach, L., Essentials of Psychological Testing . New York: 
• Harper and Ro\^^, 1969 (3rd ed..) • 

ITiis is a basic text in testing theory and practice. 

6. Jamison, D.T., and Klees, S., The Cost of Instnactional Radio and 
'television for Developing Countries > Stanford: Institute for 
Communication Research,, Stanford University, March, 1973. 

Basic methodology of cost analysis for technology projects, along 
with analyses of eight projects. 

7. Kerlinger, F.N., Foundations of Behavioral Research: Educational 
Psychological Inquiry . New York: Holt, Rinehart^ and Winston, 1972 
(2nd ed.j . 

A conprehensive text in research methods,. not for beginners. 

&. Mager, R.F., Preparing Instructional Objectives . Palo Alto, California: 
Pearon Publishers, 1962. 
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9. Rosonhein, B., Teacher Behavior and Student Achievement . New 
York: Humanities Press, 1971. l.E.A. Studies, No.l. 

A basic review of many studies o£ teacher behavior in class as it 
relates to outcomes in learning. Other studies worth noting are 
^ two field tests with ITV: 

Mayo, J,A. , Teacher Observation in El Salvador . Stanford: 
Institute for Communication Research, Stanford University, 
February, 1971. 

Mayo, J.A., Teacher Observation in ^fexico . Stanford: Institute 
for Communication Researcn, Stanford University, May, 1973. 

A summary of soine of the literature and some policy inplications in: 

4 

Rosenhein, B., and McGaw, B. , -"Assessing Teachers in Public 
♦ Education" in House (ed.) , School Evaluation > o^.cit. , ch.l2. 

1-0 ; Schramm, W. , "Feedback" for Instructional Television . Stanford: 
Institute for Communication Research, Stantoid Uhiversity, December » 
1969. . „ / . > 

Brief treatment of the range of feedback methods for ITV. 

!!• Tuckman, B.W, , Conducting Educational Research . New Yol^k: 
Harcourt , Brace and Jovanovich, 1972. 

A brief text on educational research jnethods; brief but meant 
for beginners in the field. 

' ^2. tfoods, D.A. , Test Construc tion. Columbus, Ohio: C.E. Merril. 
1960. 

Only an exaji5)le of many practical approaches to acliievement test 
g construction. 

HI. Plying and Evaluation of Technology 

A. Planning ' 

There have been few ITV projects that have planned and published 
plans for their projects. Some examples of plans that also include 
evaluation may be helpful to planner/evaluators. 

Korea : Systems Analysis for Educational Change: The 

Republic of Korea . Morgan and Chadwick {eds.) 
laiianassee: Florida State University Press, 1971. 

An exanple of systems analysis approach to 
planning an educational teclinology project. 

Brazil: SACf Project: Project Review Meeting . 

Washington, D.C. : NASA, April, 1972J 
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The outline of planning for major field e^q^eriment 
using educational technology in Brazil's ^ 
Northeast, (l^date of progress in English: 
Pulcherio, A., 'The SACI Project and the 
Educational Experiment of Rio Grande do Norte". 
Paper given at American Association for the 
Advancement ^of Science conference, Mexico Cj.ty, 
June 1973.) 

Ivory Coast : Ivory Coast Rep. and Unesco: Actualisation du 
prograimie d*gducation tgl6visuelle ; 1973'1976 . 
Faris/Abidjan: Ivory Coast Rep. and Unesco, 
1973. 

An update of the original planning volume of 
large ITV project for primary school in Africa. 

Evaluations 

There have been few evaluative studies of ITV or radio projects that the 
present authors have been able to find. These represent most of 
tliose available at present. 

14. Comstock, C, and Maccoby, N. , The Peace Corps Educational 
Television Project in Colombia - Two Years of Research ; " - 
Ovenaew.' Stanford: Institute tor Communication Research, 
Stanford IMversity, 1966. 

This volume summarizes 10 separate reports on an ETV project; 
most of the research concentrates on evaluating the Peace 
Corps role in the project. 

15. Homik, R. , Ingle, H. , Mayo, J.K., McAnany, E., and Schramm, W. , 
Television and Educational Reform in El Salvador: Final 
Report . Stanford: Institute for Communication Research, 
Stanford Itoiversity, August, 1973. 

Reports results of a four year conprehensive evaluation of 
El Salvador's ITV project. Most conplete published to date. 

16. Mayo, J.K. , McAnany, E., Klees, S., The Mexican Telesecundaria: 
A Cost-effectiveness Analysis . StanTorJ^ Institute for 
Communication Research, Stanford L&iiversity, March, 1973. 

Reports results of a year's evaluation of Mexico's ITV 
project to extend secondary education to rural areas; only 
evaluation to atteiipt to conpare costs of traditional and 
ITV schooling. 

17. Schramm, W. , ITV in American Samoa - After Nine Years . 
Stanford: Institute for Communication Research, Stanford 
University, March, 1973. 

First evaluation report on an early ITV project with enpirical 
data. r, 
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18. Schramm, W., Coombs, P., Kahnert, F. , and Lyle, J., Memo to 
Educational Planners and New Educational Media in Action; 

" Case Studies. for Planners , i^aris: Unesco/IIEP^ it)fi7 vnls.'t 

An overview and three volumes o£ case studies reporting on ' 

a number o£ radio and television projects in developing countries 

19. Schmelkes deSdtelo, S. , The Radio Schools of the Tarahumara, 
Nfexico: An Evaluation . Stanford: institute for Coimunication 
Kesearch, Stanford Uhiversity, March, 1973. 



20. 



Spain, P., A Report on the System of Radioprimaria in the 
State of San Luis Potosi, Mexico . Stanford: Institute for 
Lomnumication Kesearch, Stanford Iftiiversity, March 1973. 



IV. General Studie s of Educational Media 



21. Ball, S., and Bogetz', G. , The First Year of Sesame Street: An 
Evaluation . Princeton: Educational Testing Service, 1970; 

The Second Year of Sesame Street: A Continuing Evaluation (2 vols."). 
Frinceton: Educational Testing Service, 1971; Read^g with 
Television: An Evaluation of the Electric Conpany .(2 vols.). 
Frinceton: hducational Testing Service, 1973. , 

A conprehensive evaluation of two well known educational TV series 
in the U.S. Fbr comment on these evaluations cf. Yin, R.K. , The 
Workshop and the World: .Toward an Assessment of Children's 
lelevision Workshop . Santa Monica: Hand Corporation, October, 1973. 

y 

22. Chu, G. , and Schramm, W. , Learning from Television: What the 
Research Says . Washington, u.L. : National Association of Educational 
Broadcasters, 1968. 

The most conprehensive review of ITV research to 1967 that is in 
wide circulation. 

23. Jamison, D.T., Suppes, P., and Wells, S., The Effectiveness of 
Alternative Instructional Media . Stanfordl Institute for 
Communication Research, Stanford University, March, 1973. 

A review of research literature on effectiveness of traditional 
teaching, instructional radio, programmed instruction, CAI and ITV. 

24. Schramm, W. , Big Media. Little Media . Stanford: Institute for 
Communication Research, Stanford University, March, 1973. 

Conprehensive review of uses of technology for education in 
developing countries, including chapters on open learning systems 
and lower cost technology as well as ITV. 

25. Schramn, W. (ed.). Quality in Instructional Televisi on. Hawaii: 
I-ast-West Center Press, i972. 
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APPENDIX A: Sample Instruments 

\ 

A. introduction 

The inclusion of sample measurement instruments below should 
be understood within the context of the preceding remarks on "What 
should be evaluated?". We have stated that there are'^many aspects of 
a television project that could be evaluated: the evaluator must decide 
which aspect(s) he or she will research. Thelsampie of instruments 
presented here will not touch all possible aspects either of the research 
we have done or of the research others might wish to do. Moreover, it 
would be irresponsible to suggest that the sample instruments presented 
below are appropriate for use by others. The selection and development 
of evaluative instruments must be guided by considerations of goals by 
cultural appropriateness as well as by a realistic appraisal of the 
methodological problems of validity and reliability. Such problems make 
us hesitant to suggest this appendix as an easy "how to" approach. 

What we hope to do ifs to present to readers instruments that 
have been used for different purposes fco give them a more concrete notion 
of how Various aspects of ITV projects (attitudes, achievement, 
teacher behavior, eta) have been evaluated in the past. 

B. Research Process 

Evaluators need to understand the broad range of options for 
examining television before deciding on the particular aspects that 
they think are most important to their particular project. But once 
evaluation goals are set, there is a set of procedures than an evaluator 
needs to follow. These steps are not peculiar to ITV but are appropriate 
to most types of social research: 

GO 




2. 

\ 

(1) Sampling so that the individuals, classrooms or schools sel^ted 
accurately represent the population to which one wishes to \ 
the inferences made with regard to that sample (example, is a 
conclusion that ITV is better th^an traditional instruction based on a 
given sample of students valid for the whole school system or whole 
potential school audience of ITV?); 

(2) Creating a design for the evaluation that would permit valid 
inferences about the sample with respect to research hypotheses 
(example, were differences in achievement between ITV and non-ITV 
classrooms the result of ITV, or the result of assignment of ITV to , 
classrooms with smarter students); 

(3) Conceptualizing, Explicating, and operationalizing variables in 
the research task (example, arriving at a satisfactory idea of 

what "urban migration" means and then choosing appropriate indicators 
in the context of tjie phenomenon) ; » 

(4) Creating valid and reliable instruments for measuring a particular 
phenomenon (example, although there is much concern with self esteem 
and attempts to create measures of it, few examples of a reliable 
measuring instrument exist); 

(5) Carrying out the data gathering in such a way as to guard against 
bias or error (example, even when a reliable and valid instrument 
for measuring teachers attitudes exists, interviewers may use it 
incorrectly) ; 

(6) Having an appropriate way of analyzing results so that valid con- 
clusions can be made (example, how can the influence of ITV be 
shown to be related causally to gap closing of achievement between 
rural and urban students) . 
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These steps schematically sununarize an unwritten but essential 
essay on the process of evaluation. All of these steps are assumed in 
creation or adaptation of instruments. ^ (The bibliography, ^specially 
parts I and II, is concerned with these questions). 

Readers will correctly note that evaluation research of the 
kind we are dealing with here has been largely generated and used in 
specific settings; we do not suggest that these are the only approaches 
appropriate to satisfy evaluation needs. Secondly, not all suggested 
areas of research need generate quantitative data. There is often a need 
to gather qualitative data to understand a television project's success 
or failure (example, administrative history or the political aspects 
of using technology) . We give no examples of the latter but for 
reference we suggest readers consult Mayo and Mayo, An Administrative 
Histoi^r of El Salvador's Educational Reform (Research Report No. 8, 
November 1971; Stanford: Institute for Communication Research). 

C. Instruments 

The following pages describe a range of instruments used 
in our vork on television projects in El Salvador and Mexico. They 
represent the /following areas of evaluation: cognitive achievement, 
program series feedback among teachers, ITV and other social attitudes 
of students and teachers, demographic questions for teachers and students, 
classroom observation, school-community inventory, student follow-up, 
and parent aspiration for students. Although cost analysis is an important 
aspect of good evaluations, there is no available sample instrument 
that would give readers a proper idea of the effort required. Readers 
will observe in Jamison with Klees (bibliography, II, 6) and Mayo, McAnany 
and Klees (bibliography III, 16) that cost analysis depends on a variety 
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of methods including some survey questionnaires. 

1. Cognitive Achievement (Mexico) / 

Achievement testing is one of the most common concerns in ITV 
evaluations. Sometimes test and measurement experts are available, often 
they are not. The tests used in the Mexico Telesecundaria evaluation 
(Mayo, McAnany, Klees, 1973) were produced by different groups within the 
Secretariat of Public Education. Those used in El Salvador were prepared 
by the Educational Testing Service at Princeton, New Jersey based on 
Salvadoran curricula, and later by evaluation personnel in El Salvador. 
As various repori s cited in III.B of the bibliography show, analysis 
of achievement data should be done in terms of a series of demographic 
and environmental factors to see not only whether the group with ITV learns 
more or less than others but also what kind of students benefit the most 
from new technology. No example is reproduced here since such tests 
are available almost universally, and in general are not specific to 
ITV projects. 

2. Attitudes of students (Mexico) and teachers (El Salvador) 

Student and teacher attitudes are often a concern of admini- 
strators of ITV projects. We distinguish here between general attitudes 
concerning the use of television in the classroom and opinions about specific 
TV series (in section 3, below). Again, the evaluator must define what 
goal he/she has in mind for trying to measure attitudes before creating 
an instrument. The instruments presented bel^ (pp. 12-13 : questions 21-23, pp. 28-29 : 
questions 42-48, pp. 33-34 : questions 1-14) were appropriate to research poals of two 
concrete projects. Readers can see how they were used in the several reports 
on El Salvador (final summary report, Hornik, Ingle, Mayo, McAnany, Schramm, 
1973) and Mexico (Mayo, McAnany, Klees, 1973). 
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3 • Tea chers ' and stu dents * opi nions about specific TV series 

Asking teachers or students what they think about a specific 
TV series is a common and seemingly straightforward task in many ITV 
projects. Problems of creating a good instrument and reliably gathering 
and summarizing data are formidable, but our knowledge of many such 
projects suggests that making this information useful for program modifica- 
tion and improvement is even more difficult. The latter problem was not 
solved in El Salvador or in Mexico; nevertheless, the following instruments 
(pp,29~31 : questions 49-58, and pp. 40-42, as well as some exploration of uses in Chapters 
3 and 6 of the final El Salvador report. Bibliography III, 15) may be 
useful to readers. 

4. Social attitude and behavior of students and teachers 

Attitude questions such as those referred to above are specific 
to ITV projects, obviously enough. However as the text makes clear, 
it is often valuable to utilize a much broader range of social attitude 
and behavior questions in evaluating outcomes of ITV or other media-based 
projects. 

Student questions may refer to aspirations (pp. 14-19 : questions 34-57, and 
pp,24-26 : questions 20-32) or to other specific social attitudes (pp. 27-28 : 
questions 33-41) . Behavior of interest may include access to and use of mass media 
outside of school (pp. 10-12 : questions 10-20, and pp. 22-25 : questions 12-17). 

For teachers, one might want to include questions concerning 
general attitudes toward teaching and education (pp. 34-35 : questions 15-27) and 
about specific aspects of the schools (pp. 37-38 : question 29). 

5. Demographic and other background information about students and teachers 

Background information about students and teachers (and their 
schools and communities. - see below) can prove highly useful in under- 
standing both the context of an ITV project, and its success or failure. 
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Use of such information has been extensive in analyses and interpretations 

of both Mexican and El Salvadorau project data (cf . Bibliography III, 15 

and 16), Instruments which can be used to gather the inf9rmation will 

vary; wording and approach will depend on the specific cultural context. 

Both students (pp. 9-10 : questions 1-10, and pp. 20-22 : questions 1-11) and teachers (pp 

38-39 : questions 30-33) may be asked to supply such background data. However, probing 

too deeply into personal matters may be unacceptable, whatever the beneficient use 

the researcher plans to make of the data* Primarily, ethical questions 

concerning invasion of privacy are properly raised. In add'ition, barging 

insensitively into intimate preserves may create resistance and thus 

affect the validity of responses to these specific questions, and to the 

questionnaire as a whole. 

6* Classroom observation 

There are a number of reasons for conducting systematic 
observations of what is going on in ITV classrooms. Trying to observe 
in a systematic and reliable way is a first goal. Trying to describe what 
the teacher and students do is one level of the effort. A further goal 
is to relate this observed behavior to how well students learn (achieve- 
ment) . The form printed below (p*43) has been used for both tasks 
(as reported by Judith Mayo in Teacher Observation in Mexico (cf . 
bibliography, II, 9)). In addition to questions of the appropriateness 
of this form to different ITV projects, readers should note that the 
administration of the form demands careful training in order to obtain 
reliable data* 

7. School-community inventory (El Salvador) 

School and community environments often differ widely within 
national or regional educational systems. One argument favoring ITV is 
that this is one resource that is shared equally by all schools. To 
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understand the influence of environment on learning, the evaluator needs 
appropriate measures of factors that differentiate schools and communities. 
The instrument below (pp»44-5Q) was used in El Salvador to gather data on 
what were thought to be important factors in the learning environment. 
The data collected were not useful in raw form for making important 
conclusions about learning and the role of television; it was only after- 
ward at the analysis stage that appropriate indexing could bring out the 
full import of the data (cf. Chapter 5, final El Salvador report). 
8» Student follow-up study (El Salvador) . ' 

The study of what happens to graduates of any school system 
is a critical question bearing on the external efficiency of the system. 
In both Mexico and El Salvador limited efforts were made to follow 
graduates of the respective ITV systems. What is important about the 
problem is not the instrument itself (thus no instrument is reproduced 
here) but rather the tasks of sampling the population and gathering the 
data at a reasonable cost. Neither problem was satisfactorily solved in 
the two projects mentioned above, but in El Salvador a large sample of 
graduates was located at considerable cost in effort and time. 
A report on the results of the follow-up study can be found in H. Ingle 
et al. , Follow-Up Study on the First Group of Ninth Grade Graduates 
(Stanford, Institute for Communication Research, 1973). 
9 • Parent aspirations for their children (El Salvador) 

What education and careers students aspire to has important 
consequences for their country. Traditionally, the study of aspirations 
has been carried out solely on students (as reported in both Mexico 
and El Salvador studies). It was felt that students' aspirations were 
a result of a number of influences including school (ITV), teachers, 
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and perhaps most importantly parents. Reaching parents to ascertain 
their ideas of school, ITV and their children's future is ..ot an easy task. 
Again, the creation of a valid and reliable interview instrument 
(pp, 51-63) is perhaps less difficult than reaching a good sample of parents 
and analyzing results in an appropriate way. Some appropriate discussion 
of these problems is found in Chapter 4 (final report) and Chapter 5 (in 
Hornik, Ingle, Mayo, McAnany, and Schramm, Complete Report on the Third 
Year of Research , Research Report No. 10 (Stanford, Institute for Communi- 
cation Research, 1972)). 
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STUDENT QHESTIONNAIRE (EL SALVADOR, 1970) 
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Full NaPie 



Name of School^ 



Grade ^ Section^ 



INSTRUCTIONS: THIS IS NOT A TEST* THERE ARE NO CORRECT, OR 
INCORRECT ANSWERS. MiAT COUNTS IS YOUR OWN 
OPINION. PLEASE* ANSl^R WITH SINCERITY. 

Each one of the following questions has ona or more answers. Put an 
"X" in the blank that corresponds to your answer. In those cases for 
which you are asked to fill in information, do so in the apprxbpriate 
8j)ace . 



Section I : General 

. 1. Age: ^Birthdays completed 

2. Sex: Female 

Male 



3* Date of birth 



Day Month Year 

4. Write the total number of people who live in your house (including 
yourself and servants, if there are any): 

Total number 



5. Of the following people, which ones live in your house? ^ 

mother/' 
father 

brothers and sisters 

grandparents 

other relatives 

others who arc not relatives 
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6. VJhat ic your father's occupation? ^ 

/• What is your mother's occupation? 

8* Indicate your parents' -level of education: 

Father Mother 

Didn't study 

Part ,of primary 

All of primary 

Plan Basico 

Commerical course 

High school 

University 

9. How long does it take you to get to school every day? 

Less than 15 minutes 

Between 15 and 30 minutes 

* Between 30 minutes and an hour 

More than an hour 



Section II 

10. Of the following information media, which do you have at home? 

new s pa p er s 

^magazines 

radio^ # 

television 

books 

11. Outside of school, how many times did you watch television last 
week? 

- never 

one or two times 
three or four times 

, l.fi've or six times 

every day 

12. Approximately how long do you listen to the radio each day? 

never 

less than an hour each day 

^ one or two hours each. day 

three or four hours each day 

more than four hours each day 



How frequt*ntly did you rend newspapers last week? 



never 

one or two times 

three or four times 

* five or six times ' 
every day 



How frequently did you read magazines last week? 



^never 

_one or two times 
three or four times 
more than four tiir.es 



How many books did you read last year? 



__none 

_from 1 to 3 
_^from 4 to 10 
more than 10 



How frequently did you go to the movies last month? 
never 

one or two times 

three or four times '^ 



^more than four times \ 

Where do you usually see commercial television? 

in your house. 

in a friend's house 

J in the house of relatives 

in some other place 



What is your favorite commercial television program? 



What day or days is that program on? 



r 
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20. How often do you sec each one of the following program:.? 

Once or 
Every Twice a 

week month Rarely Never 

A. Tarzan . 

• B, Tom Jones 

C. Oficina para todos 

D» Tierra de gigantes 



Section III 



f 

INSTRUCTIONS: 



In this section you will find a series of statements. There are 
five possible answers for each statement that go from "Completely 
agree" to "Completely disagree." You should choose the answer that 
most closely approximates your own opinion and put an "X" on the 
corresponding line* Example: 

Playing with a ball is lots of fun* 

Completely Agree Undecided Disagree Completely 

agree disagree 

Please answer the following statements that are about Educational 
Television. Remember we want to know your personal opinions. 
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21 • You learn more during classes with television than during 
classes without television. 



Completely Agree Undecided Disagree Completely 

agree disagree 

22, Classes with television are more difficult. 
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23. The picture-quality on television is good. 



24. It is easier to understand classes with television than classes 
without I television. 



25 w Classes wich television do not give one enough opportunity to 
express his opinions. 



26. My parents know a lot about the use of television in my school. 



27* It seems that classroom teachers prefer to teach with television. 



28. It is more difficult to ask questions in classes with television 
than in other classes. 



29.* Classes with television are more enjoyable than classes without 
television. 



30. From which of the following situations do you learn most? 

from my own study 

from my courses with classroom teachers 
from written work or group projects in class 
from Educational Television programs 

31t From which of the following situations do you learn least? 

from my own study 
from my courses with classroom teachers 

from written work or group projects in class 
from Educational Television programs 
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Section IV 



32. Which subject do you most like to study? 

. M athematics 
N atural sciences 

Social studies 

'English 

Spanish 

All of the above 



33 • Which subject do you least like to study? 

^Mathematics 

^Natural sciences 

Social studies 

^English 

Spanish 

^All of the above 



34 • How far do you intend to go in school? 
^Finish Plan Basico 

^Finish a career course, after Plan Basico 

Finish high school 

F inish the university 
Specialize after graduating from the university 

35« How sure are you that you will finish the studies you hope 
to complete? ^ , 

I am certain I will not finish ^ 

I believe L will not finish 

I may finish 

I believe I will finish 

I am certain I will finish 



36# Of the following reasons, mark the most important one that you 
believe would not permit you to study as much as you want to: 

studies will be too difficult 

opposition of my parents 

lack of money 

lack of opportunity 
other reasons 

no reason 
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37. What level of studies do you consider necessary for the 
majority of the Salvadorean population? 

Primary school 

Plan Basico (Jr. High) 

Short career 



* High school 
University 



38» Who is most concerned about your education? 



^father 

pother * 
another relative 

_^another person who is not a member of the family 
no one 



39. Which career would you most like to follow when you finish 
your studies? 



40. The career noted by you in the previous question was chosen 
by you for which of the following reasons: 



} 



^it pays a good salary 
^it is a respected career 
^that career is one that helps other people 
it is a "short" career 

jyoii prefer it, but for no particular reason 
other reasons 



41, If for some reason you are unable to have the career you 

selected in Question 39, what kind of work will you probably 
do? 



42. What career would your parents most like you to have? 



43. How frequently do you talk to your parents about the careers 
you might have? 

^ never 

from time to time 

frequently 
very frequently 
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44. When you finish your studies, with whom would you like to work? 

the government 

a large company 

a small company 



« on my own 
with someone in my family 



45. When you 'finish your studies and begin to work, where would 
you^ like to live? 

in a small town 

in a city other than San Salvador 

in San Salvador 
outside the country 



46. When you finish your studies, would you be willing to live 
and work in a small town? 

. completely willing 

^more or less willing 

^more or less unwilling , 

completely unwilling 



47. What monthly salary do you beYieve is necessary to live 
decently? , 

V from 100 to 200 colones (1 colon equals $.40 U.S.) 

from 200 to 300 colones 

from 300 to 400 colones 

from 400 to 500 colones 

from 500 to 600 colones 

more than 600 colones 



48. Do you work in addition to attending school? 

work for a salary outside of the home 

w ork with parents or relatives and receive a salary 

^work only on household chores 

do not work regularly 



17. 



49. llf at the end of Plan Basico you were to be offered a good 
paying job but one that would not permit you to continue 
jour studies, would you take the job? 



Yes 
"*No 



Undecided 



50. What kinds of things do you like to do most in your spare 
time? 



1* 
2. 
3. 



Section V 




51. How will life be for the majority of students in your class? 

^very similar to that of their parents 

almost like that of their parents 

generally different from that of thdir parents 

very different from that of their parents 

52. What is nhe best way to get ahead in a job? 

to be intelligent 

to work harcl 

to work a long time In the same place 

^ to know how to work well with other people 

' to have friends or relatives who have influence 
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Considei" each one of the following occupations and mark 
down whether you would be happy or unhappy to'' do that 
kind of work, 

Happy Unhappy 

A. Bookkeeper 

B. Day- laborer 

C. Engineer 

D« Small farmer 

E. Brick- layer 

F. Doctor 

6. Industrial technician 

H. Chauffeur 

!• Lawyer 

J. Accountant 

K. Architect 

Electrician 

M. High school teacher 

N. Nurse - 

0* Bilingual secretary 

P. Insurance agent 

Q. Primary school teacher . 

R. Business manager 

S. Soldier 



19. 
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54. Getting a good education is worth the sacrifice o£ being 
away from one's family. 



Completely Agree Undecided Disagree Completely 

agree disagree 

55. In general, it is better to accept a good job when it is 

offered, rather than continue\one*s education with the hope 
of getting a better job in the^^^uture. 



56. Did you ever have to repeat a grade? 
yes 



no 



\ 



57. VThat career do you think is most importaixt for the development 
of El Salvador? 



{ 6 
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'STUDENT QUESTIONNAIRE (MEXICO, 1972) 



Student: 



School: 









Maternal Name 


First Name 



School Name School Number (Key) 



Home Address: 

Street Number Postal Zone 



Town Municipality Stat0 

INSTRUCTIONS: THIS IS NOT A TEST, IT IS A QUESTIONNAIRE; THERE 
ARE NO CORRECT OR INCORRECT ANSWERS; WHAT 'MATTERS 
IS YOUR OPINION, SO ANSWER SINCERELY • 

I 

Each of the following questions has one or more answers*/ Make 
an "X" on the blank spaces that correspond to the answei^fS you 
select. When you have to complete information that is i^equested 
write on the space provided • Answer ALL questions. There is no 
time limit, but don!t spend too much time on any one question. 



Year 
house? 



GENERAL INFORMATION SECTION 

1 . Age : 

2. Sex: Female 

Male 

3. Date of Birth: 

Day Month 

^1. Of the following people, which live in your 

mother 

father 
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^1. (continued) 

^ brothers or sisters 

grandparents 

other relatives 

. others who are not relatives 

5* Write the total iiiunber of people that live in your house 
(including yourself): 

total number of people 



6. V/hat kind of work does your father do? 
7* What kind of work does your mother do? 
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8. Indicate the highest level of schooling attained by your 
parents : 

A. Your Father B. Your Mother 

Never went to school Never went to school 

Studied part of primary Studied part of primary 

Completed primary school Completed primary school 

Junior high school Junior high school 

Commercial school Commercial school 

Academic high school Academic high school 

University University 

Don't know Don't know 

9. How much time does it take you to get to school each day? 
less than 15 minutes 

between 15 and 30 minutes 

between 30 minutes and one hour 

more than one hour 

80 



22. 



10. Do you work as well as attend school? (Mark only one 
answer; the one that matches the activity that occupies 
most of your time after schools) 

work for pay outside my household 



work with my parents or other relatives and am 
paid 

work only on household chores 
I don*t work 



11, If you work for pay, about how much do you earn each month? 
pesos 



SECTION II / 

""■^^ following media do you have at home? (Make an 

X alongside those which you have at home) . 

^ newspapers 

^ magazines 

radio 

television 

books 

13. About how much time do you listen to the radio each day? 
never 

less than one hour each day 

1 or 2 hours each day 

^ 3 or il hours each day 

more than k hours each, day 
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How often did you read newspapers last week? 

never 

1 or 2 times 

, 3 or ^ times 

5 or 6 times 

every day 

• 

How many books did you read last year? (Do NOT include 
comic books and school textbooks) . 

none 

one book 

about 2 or 3 

between i\ and 10 

more than 10 



How often did you go to the movies last month? 

^ never 

once or twice 

3 or ^ times 

more than 4 times 

Outside of school, how many times did you watch televisioi 
last week? 

ne^ei- 

once or twice 

3 or ^ times 

* 5 or 6 times 

every day 
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SECTION III 

18. What is your favorite subject in school? 



19. What is your LEAST favori^-e subject in school? 

20. How far do you want to go in school? 

Finish only Secondary School / 

j; Finish Teachers College or a Commercial Course 

after Secondary 

Finj.sh Academic High Sctibol (Preparatory School) 

" Finish the Univera^ity or Polytechnical School 

Specialize as a graduate student of the University 

or the Polytechnical School 

21. How sure are you that you will finish the studies you hope 
to complete? 

I am certain I will not finish 

I believe I will hot finish 

I may finish 

I believe I will finish 

I am certain I will finish 



22. Would you^be willing to move away from your family in order 
to continue your education in the future? 



yes 
no 



23* What career would you like to enfcer when you finish your 
studies? 



8.< 



25. 



What is your main reason for selecting this career? 



If for some reason you are unable to have the career you 
selected in Question 23, what kind of work will you 
probably do? 



What do your parents think about ^^your plans for the 
future? 



they are in complete agreement with me 
they are more or less in agreement with me 
they are more or less in disagreement with me 
they are in complete disagreement with me 
I don't know what my parents think 



In general, what kind of Job viould you prefer in the 
future? 



a secure Job, but one without the opportunity to 
advance 



a Job with a good opportunity for advancement, but 
secure 



If at the end of Ninth Grade you v/ere to be offered a 
good paying Job but one that would-not permi-t you to 
continue your studies, would ^rou take the Job? 

Yes 



No 

Don't know 
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29* What monthly salary do you believe is nec,escary to live 
decently? 

from 500 to 1000 pesos (1 peso equals $.08 U.S) 

from i,000 to 1,500 pesos 

from 1,500 to 2,000 pesos 

from 2,000 to 2,500 pesos 

more than 2,500 pesos 



30. When you finish your studies, would you be willing to 
live and v/ork in a small town? 

completely willing 

more or less willing 

more or less unwilling 

completely unwilling 



31. When you finish your studies and begin to work, where 
would you like to live? 

in the countryside 

.in a small town ^ 

____ in a city other than the capital 
Mexico City 



32. When you finish your studies, with whom would you like 
to work? 

the government 

a large company 

a small company 

on my own 

with someone in my family 
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SECTION IV 

In this section you will find a series of statements. There 
are five possible responses to each statement > which run from 
"Completely Agree" to "Completely Disagree." You should choose 
the response closest to your personal opinion and make an "X" 
above the corresponding line. Example: 

PLAYING WITH A BALL IS LOTS OP PUN* 

X ' 

Completely Agree Agree Not Sure Disagree Completely Disagree 

33. WHEN A PERSON FINDS A WELL-PAYING JOB, HE SHOULDN'T CHANGE 
FOR ANY REASON. 



Completely Agree Agree Not Sure Disagree Completely Disagree 



3^1. IN GENERAL, LIFE IS BEST IN SMALL TOWNS WHERE A PERSON KNOWS 
EVERYONE . 



Completely Agreed Agree Not Sure Disagree Completely Disagree 



35. THERE ARE SO MANY THINGS I DON'T HAVE THAT I WOULD LIKE TO 
HAVE. 



Completely Agree Agree Not Sure Disagree Completely Disagree 



36. ONLY THOSE PEOPLE WHO CAN CHANGE THEIR IDEAS AS THE TII^ES 
CHANGE CAN BE SUCCESSFUL IN THE MODERN WORLD. 



Completely Agree Agree Not Sure Disagree Completely Disagree 



37, WHEN A PERSON HAS FOUND A SECURE JOB, HE NO LONGER HAS TO 
WORRY ABOUT LEARNING NEW THINGS. 



Completely Agree Agree Not Sure Disagree Completely Disagree 
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38. THERE IS SO MUCH TO DO IN THE WORLD iTHAT IT WOULD BE BAD 
TO LIVE ONLY IN ONE PLACE THROUGHOUT ONE'S LIFETIME. 



Conft)letely Agree Agree Not Sure Disagree Completely Disagree 

39. I PREFER TO HAVE A WELL'-PAYING JOB TO ONE THAT FULFILLS MY 
PERSONAL INTERESTS. 



'Completely Agree Agree Not Sure Disagree Completely Disagree 

llO. IT REALLY ISN'T IMPORTANT FOR A PERSON TO KNOW INTERNATIONAL v 
;^EWS, SINCE IT DOESN'T AFFECT ONl^'S LIFE. 



completely Agree Agree Not Sure ♦ Disagree Completely Disagree 
ill. PEOPLE SHOULD BE SATISFIED WITH WHAT THEY HAVE. 



Completely Agree Agree Not Sure Disagree Completely Disagree 



SECTION V: TELESECUNDARIA (Answer these statements in the same 

way you did the previous section) . 

^2. THE PICTURE-QUALITY ON TELEVISION IS GOOD. 



Completely Agree Agree Not Sure Disagree Completely Disagree 
^3. CLASSES WITH TELEVISION ARE DIFFICULT. 



Completely Agree Agree Not Sure Disagree Completely Disagree 

THERE IS NOT SUFFICIENT TIME TO ASK QUESTIONS OR OFFER 
OPINIONS AFTER THE TELEVISION CLASSES. 



Completely Agree Agree Not Sure Disagree Completely Disagree 
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115. MY PARENTS LIKE THE PACT THAT I RECEIVE TELEVISIOK IN 
m SCHOOL. 

Completely Agree Agree Not Sure Disagree Completely Disagree" 

116. IT SEEMS THAT CLASSROOM COORDINATORS PREFER TO TEACH WITH 
TELEVISION. 

Completely Agree Agree Not Sure Disagree Completely Disagree 

1|7. IT IS EASY TO CLARIFY DOUBTS IF I DON'T UNDERSTAND SOME- 
THING ON TELEVISION. 

Completely Agree Agree Not Sure Disagree Completely Disagree 

1|8. I WOULD PREFER TO STAY IN TELESECUNDARIA EVEN IF I HAD THE 
CHANCE TO GO TO A REGULAR SECONDARY SCHOOL. 

Completely Agree Agree Not Sure Disagree Completely Disagree 
SECTION VI: THE TELELESSONS 

Instructions: Answer the following questions by marking an 
"X" next to the statement that is closest to your ovm opinion. 

hS, What do you think about the telelessons in: Mathematics? 

I like them ^ 

l/iie±tiiev like nor dislike them 

I do not like them 

50. What do you think about the telelessons in Spanish? 
I like, them 

I neither like nor dislike them 

I do not like them 

8o 
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51. V/hat do you think about the telelessons in Physics? 
I like them 

I neither like nor dislike them 

I do not like them 

52. What do you think about the telelessons in English? 
I like them 

I neither like nor dislike them 

I do not like them 

53. What do you think about the telelessons in Chemistry? 
I like them 

I neither like nor dislike them 

_____ I do not like them 

5^. What do you think about the telelessons in Current Events? 
I like them 

I neither like nor dislike them 

I do not like them 

55* What do you think about the telelessons in Vocational 
^ Activities? 

I like them 

I neither like nor dislike them 

I do not like them 



56.' What do you think about the telelessons in Civics? 
I like thera 

_____ I neither like nor dislike them 
I do not like them 
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57* What do you think about the telelessons in Physical 
Education? 

I like them 



I neither like nor dislike them 



I do not like them 



58. What do you think about the telelessons in Music 
Education? 

I like them 



I neither like nor dislike them 



I do not like them 



List of Occupations , 

01 unemployed 

02 retired 

03 self-employed 



Level 1 

10 
11 
12 
13 

in 

15 
16 

17 
18 



bricklayer 

farmer 

driver 

merchant 

beautician 

industrial worker 

photographer 

day laborer 

policeman or soldier 



19 mechanic 

20 tradesman 

21 machinist 

22 tailor 

23 servant 

25 telephone operator 

25 waiter 

26 other occupations 



Level 2 

31 
32 
33 
3^1 
35 
36 
37 
' 38 
39 



salesman 
accountant 
graphic artist 
practical nurse 
social worker 
keypuncher 
teacher 

military officer 
newspaperman 



1|0 pilot 

^11 radio technician 
l\2 secretary 
i|3 technician 

extension agent 
1|5 medical technician 
i|6 - . master mechanic 
117 \ other occupations 



90 



32. 



Level 3 



f i. 


lawyer 




musician or artist 




agronomist 




oceanographer 


73 


architect 


85 


psychologist 


7^ 


biologist 


86 


chemist 


75 


dentist 


87 


sociologist 
veterinarian 


76 


diplomat 


88 


77 


economist 


89 


engineer 


78 


civil engineer 


90 


professor or secondary 


70 






e/iV^/^/^T 'f" ^ o /I ^ vi 

ocnooj. tccicncx 


80 


pharmacologist 


91 


doctor 


81 


physicist, mathematician 


92 


high-ranking military 


82 


business manager 




or police officer 






93 


other professions 



i 
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CLASSROOM TEACHER SURVEY (EL SALVADOR, 1970y\_ 



Section I ; Educational Television (ETV) 



Students learn more with ETV than wi 



thout it. 



Completely 
agree * 



Agree 



Undecided 



Disagree 



Completely 
disagree 



2. It is more difficult to maintain classroom discipline when using 



3. Classroom teachers improve their methods by watching the 
teleteacher. 



4» ETV diminishes the importance of the classroom teacher. 



5. ETV classes are an obstacle to the interpersonal relations 
between the classroom teacher and and his students. 



6. Students learn to study better by themselves when they receive 
their classes by ETV. 



7. Classroom teachers learn to organize their schedules better with 
the ETV system. 



*A11 of these questions (1-27) have the same five alternatives. 



ETV. 
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8. Thero is a serious obstacle to learning by ETV because students 
cannot ask questions until the program has ended. 



9. It is possible to teach more with ETV during the year, because 
ETV can cover more material* 



10# Instruction by ETV makes the student more passive in class. 



11. The ETV schedule does not allow enough flexibility for the class- 
room teacher to teach his material. 



12. ETV helps parents become more inucr^sted in the education of their 
children. 



13. Instruction by ETV give$ information, but it cannot transmit values. 



14. Students would learn more if they didn't have ETV. 



Section II : Teach^iag^^d Education 

15. Teaching is not a profession that gives much satisfaction. 



16. All youngsters should have the opportunity to finish Plan Basico 
(Jr. High School). ; 



17. Increases in enrollment reduce the quality of secondary education. 



18. The fundamental goal o£ education is to form the character of the 
child. ^ 
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19. I would encourage my best students to become teachers. 



20. Only the best students should continue studying after primary 
school. 



21. In El Salvador, teachers are onuch respected. 



22. The majority of jr. high school students is notVvery interei^ed in 



XlIC lllt-kj ^A- J. w*. J • c?"" ^ ^ \ w 

learning. 



23. I would remain in education even if I found another job with »a 
better salary. 



24. Many students do not respect their teachers* 



25. The most important goal of education is to develop reasoning. 



26 • The great majority of students is motivated to make good use of 
jr. high school education. 



27. The current Educational Reform is moving toward high quality jr, 
high school education. 



ERJC 



94 



36 



Sc?ct:ion III : The Prestige of Occupations 

28. Please indicate your idea of the prestige of each one of the 

following occupations (mark your answer with an '*X" in the 
appropriate space) ♦ 

^ Very Aver- Very 

- hifih ' Hifih a^e Low Low 

A. Bookkeeper 

B. Day-laborer ' 

C. Engineer 

D. Small farmer 

E. Brick- layer 

F. Doctor 

G# Industrial technician 

H. Chauffeur ^ - 

I. Lawyer 

J 4 Accountant 

K. Architect 

L. Electrician 

M* High school teacher Jt7 



N. Nurs^Je 



0. Bilingual secretary 

P. Insurance agent 

Q. Primary school teacher 

R. Business manager 

S. Soldier 



V 
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Section IV : Problems in Education 

29 • According to your personal experience, please indicate how 
you consider each o£ the following problems, answering with 
an • 

f 

Problems in the Classroom 
A. Guides and workbooks don't arrive on .time. 



Very serious Serious Minor Very minor 

Lack of teaching materials. 



Too many students in class. 



D. Poverty of the students and their environment. 



E. The behavior of students. 



F. Technical problems in the reception of teleclasses. 

7"~'^ ^'— — 

Problems in the Educational System 

G. Lack of supervision. 

\ 



H. Lack of parents' cooperation. 

1 



The* ocoiiomic situation of teachers. 





J. School administration. 



K. The efficiency of thG Ministry of Education, 



L# Lack of teachers* with a "vocation" for teaching. 



1 

11, phanges in^the system of evaluation and promotion. 



N. Method of appointing "teachers^ 



Section V; Peirsonal Data 



30* Birthplace: City_^ 



Department of^ 



Do you reside in the 'city where you teach? Yesj N o . " 

If you answered "tJo" above, where is your permanent residience? 

City^^ [ - ' 

♦ ^ Department* ofj ^ 



Age ^ * Sex: *Male^^^|;^ Female^ 



31. dark. in one of the following blanks how long you^have been 
/teaching with ETV: 

First year I've taught with ETV r 

Second year I\e taught with ETV 

I don't teach with ET\^ 
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32. Mark ia the blanks your classification as primary school 
teacher, if you have one, and for the other levels tnark 
only those you have graduated from, except university • 

r ^ . ' 

Teacher dl assif ication: 

■ — \ — ' I — — », ^ 

^Class B C>ass A and High school 

^* * C lass A , /C lass A and Accountant 

^Iligh school* ^ / H igh school and Accountant 

^Accountant \ Class A, High school and 

Accountant 



Higher education ; 




_No higher education ^ 

^Superior Normal 

^1-2 years at the university 

__3 or more years at the university 



33. Date when became ci teacher 

; Date when became^ a secondary teacher 



34. Mark with an "X"- the subjects you teach. 

• ^Mathematics 

Natural science . 

. / Social studies 

* I ' ' Spanish ^ \ 
English 

f V 
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CLASSROOM TEACHERS' FEEDBACK SURVEY (EL SALVADOR, 1970-1972) 



INSTRUCTIONS: 

> 

The following material refers only to the MatH* course at the grade 
level indicated. Please consider only this course when giving your 
answers » 

For each of the 'following questions you should r^espond in the 
following* manner: 

If the statement is: "The ability of most students to learn 

Mathematics** 

'12,345 
you should decide whether you think that ability is high or low. If 
you think it is very high, you should make a circle around the number 
**5". If you think it is very low, you should make the circle around the 
number "1". If the ability is mid way between fiigh^and low, "3" would 
be the appropriate number to circle. If it is high, but not very high, 
you should circle the number "4". If it is low, but not very low, you 
should circle the number "2*'* 



Learning ,^ # 

1. What students learn from mathematics with ETV# 
1 2 3 4 '5 ' 



2. What students learned from mathematics ' before the introduction 
of ETV, ' 

12.345 

\ 

Motivation 

3, The motivation of the students in mathematics since the ^ 
introduction of ETV. 

1 2 3 ' 4 5 

4» The motivatic© of students in mathematics before the introduction 
of EVJ. 

1 2 i 4 5 



* Prepared according t-o the subject and grade taught. 

911 



Guides for Teachers of. Mathematics 



5. The aid to teaching given by the guides of mathematics.- 

1 2,3 4 5 

6. The practical value of classroom activities suggested by the^ 
guides for rfnathematicst 

1 2 3 4 5 

?• The relationship between the guides of mathematics and the tele- 
classes of mathematics 

1 2 3 4 5 

*< 

The Teleteacher of Mathematics 

8. ' The teleteacher *s kno\i7ledge of mathematics. 

. ], 2 3 4 5^ 

9. The teleteacher 's ability to teach mathematics. 

1 2 3 4*5, 

10. The teleteacher *s ability to m^ke 'students participate. 

1*2345 

11. The teleteacher 's ability to teach mathematics, in j:omparison 
^ with the majority of classroom teachers. 

Much less Less Equal Much more More 

Student Workbooks for Mathematics ^ 

12. The number of exercises generally iacluded in the workbooks: 

Very I A bit Adequate A bit Very 

insufficient insufficient excessive exces 
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T^^leclasses for Mathematics 



13r; In general, the content of. the telecl^sses is: 

• I 



Very A bit . . Adequate A bit Very^ 

insuffipient insufficient ^ r excessive excessive 

14* The quantity of exposition by the teleteacher is: 



Very A bit Adequate A bit Very 

insufficient insufficient excessive excessive 

J* 

15/ The quantity of audidvisual" materials (movies a^dvs^lides) used 
in the teleclasses is: 



Very A bit Adequate ^ A bit Very 

insufficient insufficient ixcefesive excessive 

16, The legibility of graphics (drayirifes, signs, etc.^ used in the 
teleclasses is: * . 



17. What the audiovisual materials (movies and slides) contribute 
to the effectiveness of teleclasses is: 



Teaching 

18. The help that ETV could provide (at its maximum) in the teaching 

of mathematics: " ' . ' 

. 1 2 3 4 5 . 

19. The help that ETV, since its introduction, has given in the 
teaching of mathematics: 

1 2 3 4 5 
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TEACHGR OBSERVATION FORM^CNEXiCO,«A971) 6 



43. 



SCHOOL , 

OEAaiER 

GRADE"" 



SUBJECT 




NO TV 



THEME OF LESSON 



{•Specity material or activity,] ,\ 



'nui ab er 




\ TEACHER 

1." Ti^cher behavior during teleclass: 



2. Lectures 



3. Dictates 



5* Asks 



prr rn 

idinR to 



] 



^ . E^qplaLns (Responidto to Spanish question - 



2?oceaui?e 



questions 



' 6 . Asks memory questions 



7. As ks stimulus-res ponse mepory questions 



\^8. Asks opinlQn (questions 



9* Asks thought questions 



10. Asks 'for exarrples 



11* Uses blackboard 



^ *13. Realosli^ 



3 . 



12 • Uses^auclio^visua 



}?ererence 



irateri-als 



works. 



^I'l* Supervises inaLvlcluai seat-work 



,.*15'. Wo rks Individua 



y 



*l6. Supervises group woric 



1 students 



17 • Suggests group projec 



l8. Assi0is homework 



. 19. AssT 



gns Invest: 



20* Checks homework 



^5aT;ions as 



nomework 
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utner coninents; 



21. Ch ps^or 

22. Ihe teacher had p^^pared his class 

in ad vanc e 

Yes Q ,.No 

STUDENTS 

1. Student behavior during tei^class: 



»2. 
»3. 

5. 
6. 

8. 



Es^und 



Dictate 



Ask clarification questions 



Ask other questions (PfTn^o^r) 



Give opinions 



Do question-answer drills 



Do 



^JrOli; 



9. Do repe 



dVion drills 



10.^ 
.»11. 
12. 

•ill. 
»15. 
»l6. 



Translai 



izati 



ons 



Vfotfk Individ 



iua 



1^ 



Work In groups 



Qo to blackboard 



Use auALo-visual. materials 



Use textbooks 



Use re: 



erence works 
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SCHOOLS AND COMMUNITIES SURVEY (EL SALVADOR, 1971) 



The School. 



A. Building 



1. 



it is rented 



it is owned >^ the government 



2. Date of construction 



3. Construction material of the, building 

4» Design as a school: 

poor mediocre - gyd 



5. Sufficient lighting (natural or electric): , * 

sufficient . . insufficient ■ . i verv insufficient 

< • • • 4' 

7. General conditions of the b.uilding: • ' * . . .-^ 

poor ^mediocre ^ 



good 



B. Teaching Conditions 



1. The noise from some claases or "from physical edi^ation; bdthers other 
classes: * ' v ^ 



rarely 



'sometimes 



£• Sufficient" teaching materials-:^ * 

sufficient insufficient 

3. Specific materials that are needed: 



frequently. 

' very j^nsuf f icient 



4* Quality of the teaching ^materials: 



poor 



mediocre 



good 
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C, Classrooms 

1. Number of classrooms: 



2. Taking into account the current number of students tl;3t use them, 
the size of the classrooms is: 

too big too little just right 



D. Desks 



1. they are owned by, the Ministry owned by the students 

owned by the Patronato (Local sponsoring group) 

2. there are enough — not enough How many -lackin g ^ 



E. Facilities ■ 

there is a library for the teachers 

there is a library for the students 

there is a special room for the library 

there is a laboratory ^ ^ 

: ' there are bathrooms 

there is a recreation area 

there is an area for physical education 

^there is an auditorium ^ 

^there is a mimeograph machine 

' * 'A 

F. Location 

1. The climate bothers*the classes:' 

In the morning In the afternoon 

Tcarely j 

sometimes » , i 

frequently J 

very frequently ^ 
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2. The noise from outside the school bothers the classes: 

rarely sometimes frequently v^ry frequently 

\ , ' ■ ■ '. 

vG. " Admi nistration 

\ ; * ' 

1, ^Combined with an Institute (in the same building) 

^ Combined with a Primary ^School (in the same building) 



_Not combined with any other school 



2.. The number of students in the ThjLrd Cycle: 



In the morning.: -7th 8th 9th^ 



In the afternoon: 7th 8th ' 9th 



3» The number of teachers in the Third Cycle:^ 
How many teach both morning and af temoon^ 
How many teachers short are you 



\ 



How many have to teach a subject outside their speciality^ 

4. The number of classes in Third Cycle: ^ - 

In the morning:, 7th 8th^ 9th 

In the afternoon: ' 7th 8th 9th 



H; The School Director 

1. permanent poslt^onX temporary appointment 

- '\\- ' ■ 

2. Se x 

3» Experience: 

How many years of teaching experience has he had_ 



How many years experience as a director has he hdd_ 



An Institute is a senior high school 
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4, Training: 

graduate of Normal School 

^graduate of Bachillerato 



j^^graduate of the Superior Normal School 



5. Retraining: 

a year at San Andres 

three summers at^'San Andres 
one summer at San Andres 

none 

6* Domiciles 

lives ^rmanently in the community 

lives in the community only during the week 

liveg outside the community 
7., What are the major problems the school has? 



I. The Students 

1. Where do the students come' from? 

all of them come from the city 

the majority come from the city, others from the Cantones 
the majority come from the Cantones 

2. Is the mix of students different in 7th than in 8th or 9th? 

In what way is it different 

3. Do you have problems with student conduct? 

rarely sometimes frequently ^vcry frequently 



Canton is an administrative division of unincorporated rural areas 

10.6 
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4. ^^lave you had time to organize extracurricular activities this year? 

yes no 

What? kind How often 



5. Have you had more activities in past years? 
\ y es no 

If yes, why the chpnge this year ' 



6* Do all of your 7th grade students come from one or from a number of 
primary schools? " ^ ' 

" all of them come from one primary 



,J^^ majority come from one, but some come from other(s) 
_they come from a number, with: equal preparation 



^^different levels of 
preparation 



7. Have you had meetings with the parents group this year? 
yes How many time s 



For what reasons 



no 
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The ContinuniCy 

A. Available Education > 

1. Are there schools in the community where ^students can continue their 
education after Third' Cycle? \ 

there are comraercial coursers day night 



-there is a Bachillerato * da y hight_ 



\ 



there are no such schools 



2. If there are no ^such schools in the city, are there such schools 
^ in the area where students can commute every day? 



^Commercial Schools dayj ^ nighty 

^Bachillerato day nighty 

there are ho such schools 



3. What are the graduates of last year doing? 
X have looked for work 

X have looked for work and begun to go to night classes in 

the Bachillerato v the Commercial School 

% are going to the Bachillerato during the day 



are going to the 'Commercial School during the day 



X other 



4. How easy is it for the graduates of Third Cycle to find work 
appropriate to" the level oi, their education la this city? 

Men Women 

All can find work ^.^.^ — . 
The majority can find work 

SOX can find work - 

The majority can.' t find work ' 

No one can find work 
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B# Accessibility , ' . • * * ^ 

r. Hov7 long does it take .to go by bus to (Santa Ana, Sonsonate, San 
f$i Miguel, San Salvador)? . ^ 



2. Hew mapy bus&s a day come here?^ 



3. How- long does it take to go b/Wbus to San Salvador? 



4/ How many T>uses " go to San Salvador from be re every day? 

*■/•■■ 

-5. How far is it t:o the, closest paved highway? 



C • Specific Data 

1. Is there- a Bank here? 



2k Is there a movie theater here?^ 

V 

3. What is the population? 



4. How many private .phones?_ 



/ 



5. How^many registered vehicles?^ 



6. What is the percentage of paved streets in the city?^ 
?• How jnany newspapers sold? 
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/ PARENT INTERVI-EW (EL SALVADOR, 1970) 



51. 



1 



/ 

1. Name of parent : 

2 . Name of student : 

3. Name of school: 



H. Including - y how many sons and daughters do you 
have? 

Ha. How many are older than ^ ? ' ^ 



(Note: if there are sons atid daughters older than 
* y ask^ questions 4^ and 4c,) 



Hb. What are the sons and daughters older than 
^ doing now? 



m or 'f 



1, 
2. 



(Note: for the older I children that are not 
studying, ask question Uc*) ' *^ 



He, After what grade did they leave' school and why? 

; Reason for leaving 



Grade after which 
they left school 



1. 
2. 
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5* V/hat lovcl oi sUioios do you hope v;il1 comi>lotc*? 

F'lan Ik;s ico Univer<;il> 

' Carro ra Coi-ta Post-graduate studies 

Bachiller'ato Other ^ specify: ^ 

6.. V/hat advantages would " have if hV finished (answer 5 )? 

1. 

2 . ' . ■ 



Can you think of any other advantage or benefit? 

3. ■ • [ 

4. [ ' 

Is there SQme.thing that might prevent * from finishing 
(feanswer^ 5) ? ' . ' - ■ 

La<Sk of will to continue studying * 

Lack of money 

^ Poor grades 

Lack* of opportunity (distance of the school , ^etc . ) 

There is nothing to pre.vent him from finishing 

^ Other reason, specify: > 



8, What kin'd of student is your son? 

Very 'good ' ' Bad 

. Good Don't know 

Fair 



9. If had to leave the family and live in another part 
V of* the country to continue his studies after Plan Bagico' , 
would- ig>ti^ en courdge him to do so? 

- Yes , No Don't know 

(Note: if parent answers' '^yes , " ask question 9a*) 

* 

III ■ 
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9a, V/ould you bo willing to pay the extra money that 
would be necessary td> send your son to study in ^ 
another part of the country? 

Yes No Part of it ^ Don't 



10* Do you think it's possible for a young man .to study 
longer than is really necessary or practical for him 
or his parents? 

Yes • No Don't knov; 



..(Note: if parent answers "yes," ask questions 10a and 



^ lObJ 



10a. What do you think of a young man who studies 
longer than is really necessary or practical? 

I ■' 



10b. Do you think that those things could happen txx — ^ ? 

Yes No Don't ^know 



11. \go you think a lot of 'schooling is absolutely necessary 

* for success, or do you think that an ambitious and hard- 
working young man could be successful without much 
schooling? 

Schooling is n«t!essary Schooling is not necessary 

Don't know 

12, Being as realistic as possible, how sure are you that 

* will remain in school until the end ( answer ^ 5)? 

Am sure he'll stay .in school , ^ 

Am fairly sure he'll stay in school 

Am not vejry sure he'll stay in school 

Am sure he'll not stay in school 



erIc 



know 

I, 
I* 
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13c Thinking about the future a little, what kind of work 
would you most like ^> to do when he finishes his 
studies? 



16 



Why do you prefer that job for * ? 




/ 


1 





^ 15 * Do you know what job your son himself prefers? 

Yes No 

(Note; if parent answers 'V^s," ask question 15a.) 

'15a. What is the- job your son prefei:s? 



If your son wer^ offered a job with a good salary when 
he finished Plan Basico , would you encourage him to 
take the job or to continue his studies? 

^ to take the job 

to continue his studies 



to take the job and continue his studies at 
the .same time 

Don^t know / 



17. At what age, mere or less, do you think that ^ ^ 
should be ready to start work and begin supporting' 
himsilf? 
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55. 



I m now going to read you a short list of' var-ioui; kih.i 
of occupation.; Alter reading each one, please tell rn''"' 
If you would be ha{.py or unhappy if chose that kind 
of work m the futur^e. 

Occupation Happy Unhappy 

Bookkeeper 

J3ay laborer 

% ^ . 

Engineer - ' 

Agricultural technician 

Bricklayer 

Doctor . ^ y 

Industrial technician 

Chauffeur • , • 

Lawyer « \ 

Accountant 

ArcMterpt * . . - - - . 

Electrician 

f - - 

Teacher ' ... .... 



What are _^'s favorite subjects at school? 



Don't know 



Ho^ much time does ^ usually spend doing his homework 
each day? . . * 

l^ss than half an hour Three hours 

One hour- ^ More than 3 hours 



Two hours Don't know 



56. 

\ 

> t 

How frequently doc-o J_ ask you to help him v.-J-ii 
homework? 

Everyday ; Qnco a in'onU. 

Once or tvjice a week Never 

In general, what do you like most about th^ edu-aticn 
your^son is receiving at this school? -^^^^n 



Without considering the cost, what changes 'would you 
like to see made to -improve the education your son is' 
receiving at this school? 



•, No change Don't know 

Is this your first visit to school this year? - . 
^ Yes " No 

Have you had a chance to talk to some of the teachers 
about the progress of your son? 

Yes • ' No 

(Note: if parents answer "yes," ask question 25a'.) 

« 

25a. V/hat do the teachers oi?. the principal say 
about the progress of youi? son? 



Have you heard or read anything about the Educational 
Reform? 



Yes No 

(Note: if parent answers "ho," omit question 27 and 
go on to question 28 i)/ 



57. 



Could you by any clumce recall somci of the things tl^e 
Ministry of Education ^has been doing since it began 
the Educational Reform? 



Have you ever seen one of the Educational Television 
programs? 

• Yes • -No 



(Note: if parent ansv/ers "yes," ask question 2 8a.) 

28a. V/here did you see the educational television 
program? - * 



at home 
at school 

someplace else , specify : 



j 



Do you have a television set at home? 
Yes , No 

Some parents think that students learn more when they > 
have classes with television; other pajpents think that 
students learn ifiore without television* Do you think 
students learn (would learn) more with or without 
television? 



they learn 'li^re with television 
they learn more without television 
don^t know 



Some parents thin]c that television classes harm stu-- 
dents* eyes;, other pa-rents think that television doesn^t 
harm students' eyes* any more than reading a book. 
What do you think? 

TV classes harm students* eyes 

, TV classes don't harm students* eyes 

dqn't.know 



\ 

\ 



I 



i 



58. 



32. Some i-ur-ant t; tliink that when students study 'with telo- ; 
vision tljcy bocoiao more alert and cntliuyiac Lie about 
studyJnsj other par'unts think-^hat " telcvi.sioa'laakcs 
, stud&nts more passive and less alert.. In y^Clr opimLon, 
"'what is (would be) the ef.fect of television? 

, ' makes students more alert and enthusiastic 



makes students more passive and less alert 
no opinion 



33. Some parents think that television makes more work for 

the classroom teacher; others think it makes less work 

for the classroom teacher. What do you think TV does 

(would do) to the" classroom teacher? 

♦ 

makes more work for the classroom teacher 

makes -le,ss work for the classroom teacher 

don^t know 



Until now educational television has been used only in 
Plan B asjco. Do you think the use of educational *ele~ 
vision sHould be extended-^: Or* should^ it remain just 
in Plan Basico ? Or would^you agree with eliminating 
it compTetely? * 

should extend the use of ETV V., 

ETV should remain just in Plan> Basico ^ 

ETV should be completely eliminated 

35. V/hy do you think that ETV should be (answer 3U) ? 
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\ 



(Note: This question is only for parents whose son- 
are in clas&es with ETV.) 

Have you noticed anything new or different in sine- 
he began t9 have classes with television? (ObtaTn 
specific answers whenever possible.) 



\ 



In his 's^tudy habits? 



In his interestis? 



T 



Is he dif j^erent from your ,o1iher children who dp not have 



classes with television? 



STOP HER E FOR A LITTLE^ Rf^T A ND ,INTaO PUCE THE NEXr SECTION 



37. How old are you? 



> p 
38. Where were you born? 



39. How long have you lived in 



**0. What opportunity did you have to study (i.e., what 
of studies did you complete?:)?, 



level 



Didn!^?t study , 

Part of primary ' . 

All of primary 
* 

^ University 



Plan Basico 
Carrera Corta 
Bachillerato 
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41. Do you feel that that level of education has been 
sufficient for you? 

Yes No Don*t know 



I 



(Note: if parent answers "no," ask questions Ula and 
41b.) - ; > 

j4la. Could you tell me why it hasn^t been sufficient? 



41b • How many more years of school would you have 
liked to complete? ' ^ 

^ Finish primary j 

Plan Basico 



Carrera Corta 



Bachillera to 
University 



42. Have you received any ^training or special courses since 
you left, school? (For example: agricultural ext^^nsion 
courises, business seminars, etc.) 



^ Yes ^ No 

1 



(Note: if parent answers "yes," ask question 4 2a.) 

42a. What kind of training or special course did you 
receive? 



43. Some parents think that if they had had more years of 
. study, their manner of^ life would be very different from 
what it is now; other parents think that more years oT 
study really wouldn^t have meant much-, tl;ieir lives would 
be more or less the same. What do you think? 

with more, studies my manner of life would be y.ery 

different 



more studies v;ouldn't have changed my manner of life 
no opinion 
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V/hat kind cif v;ork do you do? 



How long have you worked at that job? 

^ for a year or less 

_____ 2-5 years 

6-9 years 

10 years or more 

)- ■ ' 

Do you work for yourself or for someone els-e? 
myself .for someone^' else 



V/ifich of the following expressions best describes 
how you sincerely feel about your work? 



"It's a b,ad jo.b" 

"It's a job like any other" 

"It's, a good job" 

"It's an excellent job" 



Have you ever "had any other kind of work? 

••. • 
Yes 1 '' ' ' No 

m 

(Note: if^ parent answers "yes," ask question U8a 
48a, V/hat other kinds of lob have you had? 



c. 

■ \ 



What sort of work do you most like? 
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50. Has it been difficult or easy "for you to find that sort 
of work? 

c^sy - ' ' difficult neither easy ^ 

nor difficult 

51. Of all the different jobs you can think of, V7hich would 
you most like to do? ' ^ 



52. Outside of- work, what activities do you most enjoy for 
rest and relaxation? 

^ / 

. % ^ 

- • t 

i 



What activities 4o you and your family like to do to-- 
gether? ^ 



53^ What kind of work did your father do? 



5H. What level of "studies did your father complete? 

Didn'^t study ^ Plan Basico 

Part of primary Carrera Corta 

all of primary > ' Bachillerato 

^ University 

55. Being realistic, do you think your manner of life has 
been similar to that of your parents, or has it been 
very different from theirs? 

It has been. similar to that of my parents 



It has been-very different from theirs 

% 

don*t know 
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W<;tB*',Do you think your son^s manner of life will be similar 
to yours,' or very different from yours? 

Similar . Different Don't know 



57. Considering everything, do you think the opportunities 
for your son are better, worse, o^ the same as those 
you had when you began? 

his opportunities for success are better 

his opportunities for success are worse 

-, his opportunities for success are the same^ . 

no opinion 



PARENT'S OBSERVATIONS: 



I 

INTERVIEWER'S OBSERVATIONS: 
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